Android service can't connect to server after recreation only in battery saving mode

Question

I'm writing a chat app with the server and Android client written in Kotlin. I create a background service that constantly reads from the socket connected to the server and sends notifications when a message arrives. Everything works fine until user taps 'x' button and closes the app. Connection with server fails during executing cleanUp code posted below. Server had gotten EOF before service managed to send EXIT request and close streams. Then, service is recreated but when it tries to connect to the server it gets ConnectException (connection refused). It happens only when battery saving mode is on. When it's off or phone is connected to my laptop with USB and charging there's no problem.

The ss command lists that there is someone listening on the specified port, so it's not that problem. I've tried to connect in a loop, i. e. try to connect 5 times every 10 seconds, but it got refused every time. I've tried listening on two different ports, but both failed even if one of them wasn't used before. Docs say that default backlog is 50, so I guess it's not that either. I tried to set a SO_REUSEADDR flag on the server socket, but still nothing. And the strange thing is, that when service is started from the app when I launch it for the second time it can connect again. So I've created a broadcast receiver that starts the service the same way as the app in case it crashes, but it's not helping either.

I really was googling it for over a week but it's my first attempt at using both Kotlin and sockets and I'm running out of ideas. If someone has a clue to what might be going on, I'd really appreciate some help.

Here is the service onStartCommand:

override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
    activeConversation = intent?.getStringExtra(CONV_NAME) ?: ""
    login = intent?.getStringExtra(LOGIN) ?: login
    if (thread?.isAlive != true) {
        thread = thread(start = true) {
                synchronized(lock) {
                    try {
                        socket = Socket(SERVER_IP, SERVICE_PORT)
                        output = ObjectOutputStream(socket?.getOutputStream())
                        input = ObjectInputStream(socket?.getInputStream())
                        output?.writeObject(Request(START_SERVICE, mutableMapOf(LOGIN to login)))
                    } catch (e: IOException) {
                        e.printStackTrace()
                        return@thread
                    }
                }
                handleMessages() //contains input?.readObject() in infinite loop
        }
    }
    return START_STICKY
}

In onDestory() and onTaskRemoved() I call this function:

 private fun cleanUp() {
    synchronized(lock) {
        thread(start = true) {
            try {
                output?.writeObject(Request(EXIT, mutableMapOf(LOGIN to login)))
                output?.close()
                input?.close()
                socket?.close()
                nullStreams()
                thread?.join()
                println("SERVICE: thread joined")
            } catch(e: IOException) {
                e.printStackTrace()
                return@thread
            } finally {
                println("Service sends broadcast to ask for recreation")
                val restartIntent = Intent(this, ServiceRestarter::class.java)
                restartIntent.putExtra(LOGIN, login)
                sendBroadcast(restartIntent)
            }
        }.join()
    }
}

ServiceRestarter:

class ServiceRestarter : BroadcastReceiver() {

override fun onReceive(context: Context, intent: Intent?) {
    val login = intent?.getStringExtra(LOGIN)
    println("SERVICE RESTARTER: receiving restart request from $login")
    val serviceIntent = Intent(context, MessengerService::class.java)
    serviceIntent.putExtra(LOGIN, login)
    context.startService(serviceIntent)
}}

Part of my server responsible for listening:

val clientsSocket = ServerSocket(CLIENTS_PORT)
    val serviceSocket = ServerSocket(SERVICE_PORT)
    serviceSocket.setReuseAddress(true)
    println("Server socket ready!")
    println("Service socket port: ${serviceSocket.localPort}")

    thread(start = true) {
            while(true) ClientThread(clientsSocket.accept(), loggedInUsers, pendingRequests).start()
    }
    thread(start = true) {
            while(true) ServiceThread(serviceSocket.accept(), loggedInUsers).start()
    }

And ServiceThread:

class ServiceThread(val socket: Socket, 
    val loggedInUsers: HashMap<String, UserConnection>) : Thread() {

private var login = ""
private val input = ObjectInputStream(socket.getInputStream())
private val output = ObjectOutputStream(socket.getOutputStream())

override fun run() {

    var request = input.readObject() as Request
    login = request.content[LOGIN] as String 
    var userConn: UserConnection?
    synchronized(loggedInUsers) {
        userConn = loggedInUsers[login]
        if(request.action == START_SERVICE) {
            println("SERVICE THREAD: New socket conn from $login")
            userConn?.run {
                println("SERVICE THREAD: putting $login output to logged in users")
                serviceStream = output
                if(pendingMessage != null) {
                    output.writeObject(Request(SEND,
                        mutableMapOf(RESULT to SUCCESS, DATA to pendingMessage)))
                    pendingMessage = null
                }
            }
        }
    }
    try { request = input.readObject() as Request }
    catch(e: IOException) {
        println(e.printStackTrace())
        cleanUp()
        return@run
    }
    if(request.action == EXIT) { 
        println("SERVICE THREAD: Service of user $login is terminating")
        cleanUp()
    }
}

private fun cleanUp() {

    synchronized(loggedInUsers) { 
        output.close()
        input.close()
        socket.close()
        loggedInUsers[login]?.serviceStream = null
    }
}}