Problem with Hazelcast eviction the firsts entries and small size in configuration

Question

This Simple JUnit Test Always fail (Hazelcast 3.9.1):

The test class is:

public class TestHZ {

    @Test
    public void testEviction() {

        Config config = new XmlConfigBuilder(getXml()).build();
        config.setInstanceName("myTestInstance");
        HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);

        IMap<Integer, Integer> cache2 = hz.getMap("MyTest2");       
        cache2.put(123, 456);
        assertNotNull(cache2.get(123));   // <--- ALWAYS OK: conf LRU and 6000 entries

        IMap<Integer, Integer> cache = hz.getMap("MyTest");     
        cache.put(123, 456);
        assertNotNull(cache.get(123));    // <--- ALWAYS ERROR: conf LRU and 200 entries

    }

    private InputStream getXml() {
        System.out.println(MY_CONF_XML);
        return new ByteArrayInputStream(MY_CONF_XML.getBytes());
    }

And a constant for configuration with this value:

private static final String MY_CONF_XML = 
        "<hazelcast xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\r\n" + 
        "   xsi:schemaLocation=\"http://www.hazelcast.com/schema/config http://www.hazelcast.com/schema/config/hazelcast-config-3.9.xsd\"\r\n" + 
        "   xmlns=\"http://www.hazelcast.com/schema/config\">\r\n" + 
        "\r\n" + 
        "   <map name=\"MyTest\">\r\n" + 
        "       <eviction-policy>LRU</eviction-policy>\r\n" + 
        "       <max-size policy=\"PER_NODE\">200</max-size>\r\n" + 
        "   </map>\r\n" + 
        "\r\n" + 
        "   <map name=\"MyTest2\">\r\n" + 
        "       <eviction-policy>LRU</eviction-policy>\r\n" + 
        "       <max-size policy=\"PER_NODE\">6000</max-size>\r\n" + 
        "   </map>\r\n" + 
        "\r\n" + 
        "</hazelcast>\r\n" + 
        "";

I always fail to try to leave the first entry when the map configuration contains a maximum size of 200 entries.

Does someone else have this same error?

Why, when configuring with size 6000 it always goes well, and configuring with size 200 always goes wrong?

score 0 · Answer 1 · answered Oct 03 '18 at 15:58

0

I found the explanation for the problem and the solution for the tests.

What happens is that Hazelcast defaults each map into 271 parts. See: Why hazelcast has default partition count of 271 and what are the parameters to chose one?

And for map sizes below 271 it fails.

To test (JUnit) then it is best to put the number of partitions to a low number. For example 11:

@Test
public void testEviction() {

    // Only for tests
    System.setProperty("hazelcast.partition.count", "11");

    Config config = new XmlConfigBuilder(getXml()).build();

And then the test is already working correctly.

answered Oct 03 '18 at 15:58

Angel Moreno

130
4

PER_NODE is number of entries per node and is not expected to be erroneous. Can you post the detailed error message on get operation? And also, does the error happen in a multi-node cluster or only when you have a single member? – wildnez Oct 04 '18 at 15:06
When I comment the setProperty line then the following Hazelcast error line appears in the traces: **ADVERTENCIA: [180.113.65.119]:5701 [dev] [3.9.1] The max size configuration for map "MyTest" does not allow any data in the map. Given the current cluster size of 1 members with 271 partitions, max size should be at least 271.** And also the Test fails. – Angel Moreno Oct 10 '18 at 08:57
I found this website, where the same thing happens: https://github.com/hazelcast/hazelcast/issues/11646 – Angel Moreno Oct 10 '18 at 09:05

score 0 · Answer 2 · answered Oct 12 '18 at 07:11

The Github issue referred here was addressed in 3.10 but they don't seem to be related to what you are doing. For 1 member cluster, setting max_size lower than the partition count is not expected to work - not allowing put operations, reason - 1 member would be hosting all 271 partitions. When you spin up another node and partitions get distributed, then you can set a lower max_size.

Since 3.10, you are now allowed maximum one put operation per partition in a cluster with single member if max_size is set lower than partition count.

So to solve this problem - either change the max_size config or reduce the partition count or add one more member and make it a cluster.

Problem with Hazelcast eviction the firsts entries and small size in configuration

2 Answers2