2

I have a problem with neo4j database. When I try to init data, it should be just create one sample data but sometimes when I try to init data, it create double sample data. There is no trace about calling second times. This is the Config of my Neo4j

@Configuration
@EnableNeo4jRepositories(basePackages = "com.example.neo.repository")
@EnableTransactionManagement
public class Neo4jConfig extends Neo4jConfiguration {
    @Override
    @Bean
    public SessionFactory getSessionFactory() {
        // with domain entity base package(s)
        return new SessionFactory("com.example.neo.model", "BOOT-INF.classes.com.example.neo.model");
    }

    // needed for session in view in web-applications
    @Override
    @Bean
    @Scope(value = "session", proxyMode = ScopedProxyMode.TARGET_CLASS)
    public Session getSession() throws Exception {
        return super.getSession();
    }
}

This is how I called my function

@RequestMapping(value = "/initCurrency")
public ModelAndView initCurrency() {
    initializationService.initCurrency();

    ModelAndView model = new ModelAndView("redirect:/");
    return model;
}

This is the initializationService function

private String[][] currencyList = {
        { "USD", "7.5" },
        { "DKK", "1" },
        { "AFN", "1"},{ "EUR", "1"},{ "ALL", "1"},{ "DZD", "1"},{ "USD", "1"},{ "AOA", "1"},{ "XCD", "1"},
        { "ARS", "1"},{ "AMD", "1"},{ "AWG", "1"},{ "SHP", "1"},{ "AUD", "1"},{ "AZN", "1"},{ "BSD", "1"},
        { "BHD", "1"},{ "BDT", "1"},{ "BBD", "1"}
}
@Override
public void initCurrency() {
    for (String[] currency : currencyList) {
        Currency existCurrency = currencyService.findByName(currency[0]);

        if (existCurrency == null) {
            existCurrency = new Currency(currency[0], Double.valueOf(currency[1]));
            currencyService.save(existCurrency);
        }
    }
}
David Vincent
  • 634
  • 1
  • 8
  • 33
  • 1
    Perhaps you should add a uniqueness constraint on the currency code - both to prevent duplicates, and shed light on what is going wrong. – Jasper Blues Oct 20 '16 at 12:31
  • @JasperBlues I already add unique Id and prevent duplicate by using findByName. If there is exist CUrrency, its should be skip the saving code. But when I check neo4j database, There are double data with differnt unique Id. – David Vincent Oct 20 '16 at 13:22
  • 1
    What Jasper means is you should make the name of the currency unique, using something like `CREATE CONSTRAINT ON (n:Currency) ASSERT n.name IS UNIQUE` (supposing `USD` goes in the `name` property of the `Currency` label). The application wouldn't be able to create 2 nodes with the same currency name, the second one would fail and you'd find an exception in your logs. – Frank Pavageau Oct 20 '16 at 13:25
  • @FrankPavageau Ahh. thank you for the explanation. Actually Currency is one of my class, so I need to make every class in my database have an unique data, for example prperty Name, right? – David Vincent Oct 20 '16 at 13:30
  • @FrankPavageau But actually, if there is same Data, I already prevent it with my code if(existCUrrency == null). But why sometimes it can bypass that conditional and make duplicate data? – David Vincent Oct 20 '16 at 13:37
  • 1
    The entities that have a unique identifier (name, uuid, whatever) should have a constraint (which creates an index and speeds up the lookup too!). It's not the case of all entities: if entity `B` only exists in the context of entity `A`, it might not have a globally unique property. E.g. if you have `Version` nodes related to `Software` nodes, the versions are not globally unique, both `Neo4j` and `Windows` can have version `3.0`. – Frank Pavageau Oct 20 '16 at 13:43
  • 1
    Your code only prevents duplicates in the case of sequential calls to the init service. If you have concurrent calls, it's a race condition when the threads execution is interleaved: thread A checks it doesn't exist, thread B checks it doesn't exist, thread A creates the currency, thread B also creates the currency. – Frank Pavageau Oct 20 '16 at 13:45
  • Yes, i noticed that It must be cause of thread. But, I only run it once and not call the function again and only 1 thread run to executed that. – David Vincent Oct 20 '16 at 14:20
  • @FrankPavageau care to turn your advice into an answer? We can provide links to index creation docs too. (Btw, we're working on an annotation to create indexes in code - not available yet though). – Jasper Blues Oct 21 '16 at 01:16

1 Answers1

2

The only reliable way to avoid duplicates is to have an actual unicity constraint on the property:

CREATE CONSTRAINT ON (n:Currency) ASSERT n.name IS UNIQUE;

There's no way in SDN 4.[0-2] (yet) to create such a constraint from the model (there was in SDN 3.x with the @Indexed or @Indexed(unique = true) annotation), so you'll have to run the query independently (for example using Liquigraph!).

Doing a lookup to guard the creation is not enough in a concurrent environment (sequential calls are OK), because there's no lock between the read and the write, which can result in the following scenario with an interleaved execution:

  • Thread A checks the existence, doesn't find a node
  • Thread B checks the existence, doesn't find a node either
  • Thread A creates a currency node named "USB"
  • Thread B creates another currency node with the same name

Since you end up with duplicates, 2 concurrent calls are happening. A load balancer with a really short timeout and configured to retry? Activate HTTP logs, add some logging in the Spring controller or service, capture the traffic with tcpdump, etc. It will be easier to isolate the second call once the unicity constraint is active, since you'll get an exception.

Frank Pavageau
  • 11,477
  • 1
  • 43
  • 53
  • Yeah. Its because 2 threads run simultaneously, thats why Its create duplicate data. Its like your explanation. ANyway Thank you for this information, really helpful. – David Vincent Oct 21 '16 at 12:15
  • 1
    It's worth mentioning that `@Index` and `@Index(unique = true)` is supported in OGM `2.1.0-SNAPSHOT`: https://github.com/neo4j/neo4j-ogm/pull/243 – digx1 Oct 23 '16 at 02:45