Querying on Firebase Database with large data set is very very slow

Question

I use Firebase database on my Android app. Normally, it works fine. But when the database is getting larger, the query performance is getting worse. I added about 5k record on database (under "elk" and "su" nodes), then I queried on database (on "cut" and "user" nodes) but all the queries are very very slow. I defined data index on database rules but it did not work. How can I solve that problem?

Here are my queries :

// query to get the zones followed by user
FirebaseDatabase.getInstance()
                .getReference()
                .child("user")
                .child(userID)
                .child("zones");

// query to get cuts on a zone
FirebaseDatabase.getInstance()
                .getReference()
                .child("cut")
                .child(cutType)
                .orderByChild("zoneID")
                .equalTo(zoneID);

When you say very very slow. How slow do you mean ? Do you have any test results or something ? How can we know the network was fine or the phone was functioning fine there should be more concrete evidence of your problem until then we can only assume and blame firebase itself for slow functioning. Please try to generate more concrete test results with minimum, medium , large database etc. While keeping in mind various factors like network strength, phone cache performance etc. — Nishant Dubey, Sep 29 '16 at 11:30
You've included a picture of the JSON tree in your question. Please replace that with the actual JSON as text, which you can easily get by clicking the Export button in your Firebase Database console. Having the JSON as text makes it searchable, allows us to easily use it to test with your actual data and use it in our answer and in general is just a Good Thing to do. — Frank van Puffelen, Sep 29 '16 at 14:26
If something is slow, it is almost without exception a function of your network bandwidth vs the data that you're downloading. I don't immediately see anything that seems extraordinary big and your indexes look correct for the query. If you can reproduce the problem in a jsfiddle/jsbin, we can try it and see how our performance compares to yours. — Frank van Puffelen, Sep 29 '16 at 14:29
Of course I tested on different network connections, different phones, minimum and large databases. But nothing changed, the query was very slow(it took about 4 minutes). I did not blame firebase or anyone. I just wanted to learn what I was doing wrong. Then I solved the problem according to Mathew Berg's answer. Now the query is not slow, it works normally. — cimenmus, Sep 29 '16 at 15:14

score 2 · Accepted Answer · answered Sep 29 '16 at 11:31

2

If you want to continue expanding the best thing to do would be to duplicate your data in a zone reference where it knows which elk/su are a part of it. Something like this:

{
    zones: {
        elk: {
            "istan-besik": {                
                "-KSp)bL5....": true,
                ...: true
            }
        }
    }
}

That way when you want to search for all you would just do:

...child('zones').child(zoneId).child(cutType)

And then loop through those to go get each elk/su directly

answered Sep 29 '16 at 11:31

Mathew Berg

28,625
11
69
90

In fact, I do not want to continue expanding, just add new records to "elk" and "su" nodes. – cimenmus Sep 29 '16 at 11:39
So yeah expanding those. Regardless you should start doing it this way so that it remains fast. There is no orderBy's using this methodology and it will be very fast. – Mathew Berg Sep 29 '16 at 11:58
I will do you said. But i did not understand something : I do not use "orderBy" query to get the zones followed by a user, and "user" node is not under "cut" node and there is only 2 users but that query is very slow, too. What is the reason of it? – cimenmus Sep 29 '16 at 12:13
Your user query is very slow? – Mathew Berg Sep 29 '16 at 12:14
Sometimes it works normally, but usually it is very slow, too. – cimenmus Sep 29 '16 at 12:18
It should be fast, double check something else isn't slowing it down (is the user object very large?). If you can't track that down write a support ticket. – Mathew Berg Sep 29 '16 at 12:26
Sorry, I realized that user query is not slow. I thought that it is slow too, because user query and "cut" query are working at the same time. But after "cut" query finishes, I started the user query and it worked normally. I tried about 10 times, all of them worked normally. – cimenmus Sep 29 '16 at 12:27

Querying on Firebase Database with large data set is very very slow

1 Answers1

Linked