According to official document, I quote:
Gradle Daemon: a long-lived background process that executes your builds much more quickly than would otherwise be the case. We accomplish this by avoiding the expensive bootstrapping process as well as leveraging caching, by keeping data about your project in memory. Running Gradle builds with the Daemon is no different than without. Simply configure whether you want to use it or not — everything else is handled transparently by Gradle.
It means that if you see this
Starting a Gradle Daemon, 3 busy and 32 stopped Daemons could not be reused, use --status for details.
32 stopped Daemon is still present from previous builds and it may has plenty of reasons. We have already this and this in stackoverflow.
But about your issue I guess your CI runner is just inside temporary docker container, and every time docker container dispose all data in memory will be disposed too. So why do you need to use gradle daemon?
The best solution is disable gradle deamon. put this line org.gradle.daemon=false
inside gradle.properties
of your project it must be at root of your project if it's not create one.
This remark has already been discussed in this, this, this