StreamBuilder taking too long to update (or not updating at all)

Question

I use firebase authentication in my app and I have a widget called AuthGate that the app launches as its home. In AuthGate there is just a StreamBuilder that listens to FirebaseAuth.instance.authStateChanges() and returns different screens depending on the user state. When no user is logged in, the LoginScreen is returned, and when they log in through one of the methods, the StreamBuilder should send them to the HomeScreen. The same should happen when registering and when they sign out it should automatically send them back to the LoginScreen. My issue is that sometimes the changes are not detected at all, and even when they are, it takes at least a few seconds. Is there a way to make it update faster? I like this method a lot more than the way I used to do it (pushing replacements), because it's cleaner and more efficient, so I hope there is a solution. Here is the code in the AuthGate:

class AuthGate extends StatelessWidget {
  const AuthGate({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return StreamBuilder<User?>(
        stream: FirebaseAuth.instance.authStateChanges(),
        builder: (context, snapshot) {
          if (!snapshot.hasData) {
            return const LoginScreen();
          } else if (snapshot.data?.displayName == null ||
              snapshot.data?.email == null ||
              snapshot.data?.photoURL == null) {
            return UpdateProfileScreen(user: snapshot.data!);
          } else {
            return HomeScreen(user: snapshot.data!);
          }
        });
  }
}