Firebase realtime database high download usage

Question

We are using firebase realtime database to save scores for each level for users that are connected with facebook. We are still testing this feature and everything works fine, but the download usage is really really high. Every time that user opens the app, I download very small ammount of data, around 20 bytes, and aditional 5 bytes for every level that he starts. After few minutes the download usage started showing more than 100kB which is a lot and won't scale financially when we realease this to our users. Here is the data structure that we are using:

users{
  facebook_id{
    "firebase_id" : firebaseId,
    "max_level" : maxLevel,
    "stars" : numberOfStars,
    "scores" : {
      level : score,
    }
  }
}

I also did some CLI database profiling, and there was 0 Nonindexed Queries and the usage seemed correct. Here is the screenshot.

Does anyone know what might be wrong? If this is just SSL overhead(which still seems too big), there noting else we could do but setup our own server.

Answer 1

Method 1 : Store the lengthy data in seperate locations

• If this is the data structure
• Everytime the user checked his maxlevel, the underlying scores for each level gets called.
• Avoid this by splitting the summated result in seperate path

Bad Way

users{
  facebook_id{
    "firebase_id" : firebaseId,
    "max_level" : maxLevel,
    "stars" : numberOfStars,
    "scores" : {
      level : score,
    }
  }
}

Better way- Split into user details and user score details

User details

Contains only max level and number of stars
users{
  facebook_id{
    "firebase_id" : firebaseId,
    "max_level" : maxLevel,
    "stars" : numberOfStars
  }
}

User Score details

• Contains scores of levels in seperate path
• client side can check users_scores/facebook_id/scores/<levelname> exists? if exists, fetch score

users_scores{
  facebook_id{
    "scores" : {
      level1 : score1
    },
    {
      level2 : score2
    }
  }
}

• Calculate the total number of stars by using cloud functions OR
• NOT RECOMMENDED: calculate the total number of stars in the client section by reading the stars from users path, and not going through users_scores. Just update the path users_scores/facebook_id/scores/<levelname> from client itself

KEY POINT

• Rather than reading all stars in every fetch request
• Fetch the last updated stars, and update the new stars to it.
• Update the star in specific level by acessing only a single path
• Which will be empty for the first time and will contain value if he already completed the level.

Answer 2

I had the same issues in my iOS app. A 250 kilobytes database was generating an incredible amount of traffic. I did a 1-hour CLI database profiling, which showed a total of 40 kilobytes of traffic, while the Usage tab in the Firebase console for the same hour showed 20 megabytes.

After a lot of tests it turned out that Firebase fetches data from the database after the reference initialization, even without any read or write operations.

Every time a user simply opened and closed the app, the Firebase console showed that around 8 kilobytes of data were downloaded from the database. To avoid this, before the initial reference to the database, you need to set isPersistenceEnabled = false however, note that this will also disable the offline mode for the Realtime Database.

import FirebaseDatabase

class RealtimeDatabase {
    
    static let shared = RealtimeDatabase()
    
    let database: Database
    let dbReference: DatabaseReference
    
    private init() {
        database = Database.database(url: "https://your-database-url.firebasedatabase.app")
        database.isPersistenceEnabled = false
        dbReference = database.reference()
    }
    
}

Answer 3

It is possible that you retrieving a lot more data from the JSON file than you intend to. Google have laid out some guidelines for how to structure your data best so that you only affect as little information as possible.

Avoid using too many indexes. An excessive number of indexes can increase write latency and increases storage costs for index entries.

Be aware that indexing fields with monotonically increasing values, such as timestamps, can lead to hotspots which impact latency for applications with high read and write rates.

https://firebase.google.com/docs/firestore/best-practices#indexes

Also this section underneath Indexes might be of help:

Avoid writing to a document more than once per second. For more information, see Updates to a single document.

Use batch operations for your writes and deletes instead of single operations. Batch operations are more efficient because they perform multiple operations with the same overhead as a single operation.

Use asynchronous calls where available instead of synchronous calls. Asynchronous calls minimize latency impact. For example, consider an application that needs the result of a document lookup and the results of a query before rendering a response. If the lookup and the query do not have a data dependency, there is no need to synchronously wait until the lookup completes before initiating the query.

Do not use offsets. Instead, use cursors. Using an offset only avoids returning the skipped documents to your application, but these documents are still retrieved internally. The skipped documents affect the latency of the query, and your application is billed for the read operations required to retrieve them.

https://firebase.google.com/docs/firestore/best-practices#read_and_write_operations

Hope this helps you!