2

I am trying to build a backend service (FastAPI) which is connected to Firebase using Pyrebase Helper Library (as suggested by Firebase documentation). The Firebase Realtime Database has a structure of:

"myproject_db" : {
        "movies":
            "key1": {
                "movie_id": 33
                "other_id": 44
            },
        "ratings":
            "key1": {
                "user_id" : 1
                "rating" : 5
                "movie_id" : 33
            },
    }

The movies and ratings db have more than 10,000 fields.

Now my method /getRatedMovies for a specific user is described below. Even though it works , it requires way more than 30 seconds to complete. The main reason is the iteration of the movies. Seems like everytime I need to find a specific movie I need to go through the entire table. Is there an alternative solution I am missing ? Thank you.

@app.get("/getRatedMovies")
async def ratedMovies():
    try:
        ratings = db.child("ratings").order_by_child("user_id").equal_to(username).get()

        restructuredMovies: List[Movie] = []
        for rating in ratings.each():
            movieid: int = rating.val()["movie_id"]
            movies = db.child("movies").order_by_child("movie_id").equal_to(movie_id).get()
            for movie in movies:
                movieid: int = movie.val()['movie_id']
                other_id: Optional[str] = movie.val()["other_id"]
                restructuredMovie: Movie = Movie(title = movie.val()["title"], movieid = movieid, other_id = other_id)
                restructuredMovies.append(restructuredMovie)
        return {"Movies": restructuredMovies}
    except Exception as error:
        return {"error": "Something went wrong."}
Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
PoolHallJunkie
  • 1,345
  • 14
  • 33
  • My initial guess is that you're firing a lot of REST requests, which simply is going to take time. So: 1) How many ratings are there? 2) How big is each movie? 3) Any more details on where the time is going in the code you shared would be useful. – Frank van Puffelen Jan 10 '21 at 15:34
  • If you're attempting to fetch all movies that have been rated, add a field on each movie that indicates that the movie has been rated. Add an index on that field on Firebase so that you can look up movies through that field. Add a proper limit for pagination so that you don't fetch the whole table (as soon as the number of rated movies increase, that will start to approach the complete table each time). – MatsLindh Jan 10 '21 at 15:49
  • There is already index in both movies and ratings. My guess is that it retrives all the entries and then I attempt to search. As Mats suggested maybe I can paginate the entries and search in bundles instead. The time will vary depending on which bundle I find it. Can I access the count so I can splitt the bundles appropriately ? – PoolHallJunkie Jan 10 '21 at 16:40

0 Answers0