1

This morning, with some struggles (see: Upgrading a neo4j database from 2.0.1 to 2.1.3 fails), i upgraded my database from version 2.0.1 to 2.1.3. My main goal with the upgrade was to gain performance on certain queries (see: Cypher SORT performance).

Everything seems to be working, except for the fact that all Cypher queries - without exception - have become much, much, much slower. Queries that used to take 75ms now take nearly 2000ms.

As i was running on an A1 (1xCPU ~2GB RAM) VM in Azure, i thought that giving neo4j some more ram and an extra core would help, but after upgrading to an A2 VM i get more or less the same results.

I'm no wondering, did i loose my indexes by doing a backup and upgrading/using that db? I have perhaps 50K nodes in my db, so it's not that spectacular, right?

I'm now still running on an A2 VM (2xCPU, ~4GB RAM), but had to downgrade to 2.0.1 again.

UPDATE: #1 2014-08-12

After reading Michael's first comment, on how to inspect my indexes using the shell, i did the following:

  • With my 2.0.1 database service running (and performing well), i executed Neo4jShell.bat and then executed the Schema command. This yielded the following response:enter image description here
  • I uninstalled the 2.0.1 service using the Neo4jInstall.bat remove command.
  • I installed the 2.1.3 service using the Neo4jInstall install command.
  • With my 2.1.3 database service running, I again executed the Neo4jShell.bat and then executed the schema command. This yielded the following response:enter image description here

I think it is safe to conclude that either the migration process (in 2.1.3) or the backup process (in 2.0.1) has removed the indexes from my database. This does explain why my backed up database is much smaller (~110MB) than the online database (~380MB). After migration to 2.1.3, my database became even smaller (~90MB).

Question is now, is it just a matter of recreating my indexes and be done with it?

UPDATE: #2 2014-08-12

I guess i have answered my own question. After recreating the constraints and indexes, my queries perform like they used to (some even faster, as expected).

Community
  • 1
  • 1
Geoffrey Braaf
  • 254
  • 2
  • 12
  • 1
    can you share your data model, data size, and the exact queries you're running? You can check your indexes with the "schema" command, if you had legacy indexes you can check them with "index --list" in the neo4j-shell – Michael Hunger Aug 11 '14 at 17:06
  • 2
    We also have a customer who upgraded from 2.0 to 2.1 and complained about the queries going up from 100-150ms to 1500ms. Apparently, starting with an empty store and inserting the dataset from scratch into an 2.1 instance did the trick for him. – albertoperdomo Aug 11 '14 at 17:42
  • I am probably the customer Alberto mentioned just above. So to sum up, I was using 2.0.4, and wanted to upgrage to 2.1.2, what Alberto and his team made well and very quickly :) However, I was afraid when I noticed some of my main query times. I was used to execute some queries in 150ms max, and after the upgrade: more than 2000 ms ! Even 9000ms sometimes! The primary thing that came in mind was I lost all the indexes..but didn't. What I did is that I manually emptied the whole database, and populated it using pure cypher queries. Now, my queries are fast like before, perhaps faster ;) – Mik378 Aug 11 '14 at 18:21
  • 1
    Can you all provide more detail? Perhaps in the Neo4j google group for detailed discussions to find the root cause? I think I'd love to profile your queries on both databases to find out what caused this. – Michael Hunger Aug 11 '14 at 19:07
  • It's pretty hard to provide all the details I had at the moment. Queries were pretty simple and this upgrade issue happened more than once. Actually, three months ago I also wanted to upgrade to 2.1.0 and noticed the same performance issue. I didn't signal the issue since I read on the blog/forums that 2.1.0 was slightly buggy..and that might ripple on this. I think simple queries involving some relationships and indexes would easily reproduce the case. They would be fast on 2.0.X and very slow on 2.1.X unless the queries are replayed manually => workaround I did. – Mik378 Aug 12 '14 at 01:52
  • I could provide you with my database for profiling/testing purposes, but i'd rather not publish it to the public Google Group. Is that still useful? – Geoffrey Braaf Aug 12 '14 at 08:18

1 Answers1

3

Eventually, it turned out that in the process of backing up my database (in version 2.0.1) or during the migration process at startup (in version 2.1.3) i lost my indexes and constraints. Obvious solution is to manually recreate them (http://docs.neo4j.org/chunked/stable/cypher-schema.html) and be on your way.

Geoffrey Braaf
  • 254
  • 2
  • 12
  • It's weird, since using Spring-Data-Neo4j, each restart of my application involves a creation (if not existing) of indexes. Faced to this problem, I restarted my application...hundred of times :) Wouldn't SDN recreate them? Mysterious.. – Mik378 Aug 12 '14 at 09:31
  • Oh, it does that; it regenerated my indexes in case they are missing. That is not what went wrong in my particular scenario. What got lost in the backup/migration process are the *definitions* of indexes and constraints, not *just* index data. – Geoffrey Braaf Aug 12 '14 at 12:38