0

I have Neo4j v2.1.6 (default configuration) and Neo4j.rb v4.1.0. All queries are slow around 50ms. I have only 5 nodes in db.

For example:

User.find_by(person_id: 826268332)

CYPHER 47ms MATCH (n:`User`) WHERE (n.person_id = {n_person_id})  RETURN n LIMIT {limit_1} | {:n_person_id=>826268332, "limit_1"=>1}

Where can be a problem?

user1657173
  • 231
  • 1
  • 2
  • 9
  • 50ms = 0.05 sec is slow? Are you running this through the REST interface, or embedded? What performance are you expecting? – FrobberOfBits Feb 02 '15 at 17:46
  • 1
    Comparing postgresql and mongo is not really a good comparison here because you're connecting via a binary protocol, sometimes on the same machine (not over a network). When using a RESTful interface, there are several other layers of tech happening. This result you're seeing (substantially sub-second) doesn't necessarily mean anything about the performance of neo4j, it could be a result of network latency, overhead of fetching/parsing json, disk caching, or any number of other factors. You need to compare apples to apples. – FrobberOfBits Feb 02 '15 at 19:24
  • Similar query in PostgreSQL or MongoDB is faster than 1ms. It's REST (Neo4j::Session.open(:server_db, 'localhost:7474')). – user1657173 Feb 02 '15 at 19:25
  • How are you doing PostgreSQL and MongoDB query via REST? Because if you're not doing it via REST, then the comparison is useless. – FrobberOfBits Feb 02 '15 at 19:29
  • @FrobberOfBits: Sorry, I wrote 2 times same answer, second skipped your previous answer. I understand, thank you. – user1657173 Feb 02 '15 at 19:33
  • Is there an Index on :User(yeti_person_id) ? – Michael Hunger Feb 02 '15 at 20:46
  • Is that the first query or a repeated query? As Neo4j loads and parses things lazily the first execution(s) get an additional hit. – Michael Hunger Feb 02 '15 at 20:47

2 Answers2

2

That is strange. In the neo4j gem I often see simple queries run in around 1-5 ms.

For debugging, what if you did this?

User.where(yeti_person_id: 826268332).first

Also, what does this give you?

puts User.where(yeti_person_id: 826268332).to_cypher
Brian Underwood
  • 10,746
  • 1
  • 22
  • 34
  • First query was similar: 48ms. Second: MATCH (result_user:`User`) WHERE (result_user.yeti_person_id = {result_user_yeti_person_id}) – user1657173 Feb 02 '15 at 19:45
  • Is the server located on the same computer as the ruby script/app? Did you used to have a lot of nodes in it? If so, have you restarted it recently? If it's local, can you run the web console or `neo4j-shell` and try to execute a cypher command and see how fast it runs? – Brian Underwood Feb 02 '15 at 20:00
2

I'm one of the core maintainers of Neo4j.rb, along with Brian Underwood, who replied above. This is not exactly a full answer since we need to know more about your system to answer that, but I'm posting this here because it's too much for one comment.

My money is on something wrong with your DB or your system. We had a similar issue reported -- slow queries when working locally, no cause able to be determined -- for a user running Windows. See Neo4j.rb version 3.0 slow performance RoR, over 1024ms for all queries. We weren't able to pin it down. Locally, running that exact same query, I see 13ms the first time I run it and ~3ms every time after that. Indexing won't make a difference in a DB that small.

Ways to limit the chance of a problem and generally improve performance:

  • Use Ruby MRI 2.2.0
  • Use Neo4j 2.1.6 or 2.2.0
  • Use Mac or Linux, not Windows
  • Require the oj and oj_mimic_json gems in your app

You will see longer responses for a query like that if your db and app server are in two different networks.

Regarding the comment that this simple query is much faster in MongoDB and PostgreSQL: yes, it's going to be. Both of those return simple queries faster than Neo4j.rb for no fewer than two reasons:

  • The Ruby gems for connecting to those DBs do not use a REST interface, they use custom binary protocols.
  • Both of those are optimized for returning single records quickly, Neo is optimized for returning large groups of records quickly.

Before releasing Neo4j.rb 4.0, I did a ton of benchmarks against Postgres and MongoDB and found the same results: they crush us when returning single objects. (PostgreSQL is amazing technology general.) As soon as you start looking for related objects, though, things balance out, and as you add complexity, the difference becomes even more significant. I don't have any numbers to share, unfortunately, but I'll make a blog post about it sometime soon if I have some time.

Community
  • 1
  • 1
subvertallchris
  • 5,282
  • 2
  • 25
  • 43
  • Thank you, ruby 2.2 make it more than 5 times faster. – user1657173 Feb 03 '15 at 10:42
  • Oh, wow, interesting. What version were you using? Maybe jruby? – Brian Underwood Feb 03 '15 at 14:49
  • @BrianUnderwood I updated from **ruby 2.0.0p598 (2014-11-13 revision 48408) [x86_64-linux]** to **ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-linux]** I tried it on two machines Ubuntu and CentOS and it has same effect. I didn't update anything else. All queries are now faster than 10ms. – user1657173 Feb 03 '15 at 16:28
  • 1
    Good to know. I'm a bit surprised that ruby 2.0.0 was that slow. I wonder if it might have been just starting with a fresh version of ruby. Chris might know something I don't, though – Brian Underwood Feb 04 '15 at 12:17
  • I'm surprised by it, too. Travis is almost always as fast or faster with 1.9.3 than 2.2.0, so I can't see why there would be a problem with 2.0.0. – subvertallchris Feb 04 '15 at 15:04
  • 1
    I can confirm this as well on Heroku with Graphene. I had queries that ran quickly (< 100ms) in dev using Ruby 2.1, but when deployed were pushing up to around 800ms per request according to New Relic. I hadn't specified a ruby version in my Gemfile, so I specified 2.2.1, and saw a massive improvement. Prior to the change Heroku was using 2.0, so if you're seeing slow queries on Heroku make sure you are running an up-to-date Ruby. – stockli Mar 07 '15 at 04:42