0

I have an app that runs 3 threads at the same time. One thread is for stablishing a bluetooth connection between the phone and another bluetooth device (Arduino). Thread 2 plays audio incoming from another phone via bluetooh. Thread 3 records and sends audio to the other phone via bluetooth.

The audio communication works with lots of glitches if phone is trying to stablish a connection with the Arduino (when thread 1 is running bluetoothsocket.connect();). However, when phone does not try to stablish a connection with the Arduino or the connection is already stablished and thread 1 is done, then the communication is good.

Here is the code for thread 1 - arduino (this code is with a class)

public class ConnectThread extends Thread {
        private final BluetoothSocket mmSocket;
        private final BluetoothDevice mmDevice;

        public ConnectThread(BluetoothDevice device) {
            // As mmSocket is final, we use a temporary socket variable
            BluetoothSocket tmp = null;
            mmDevice = device;

            // Get a BluetoothSocket to connect with the given BluetoothDevice
            try {
                // MY_UUID is the app's UUID string, also used by the server code
                tmp = device.createRfcommSocketToServiceRecord(UUID);
            } catch (IOException e) { }
            mmSocket = tmp;
        }

        public void run() {
            android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
            // Cancel discovery because it will slow down the connection
            mBluetoothAdapter.cancelDiscovery();

            try {
                // Connect the device through the socket. This will block
                // until it succeeds or throws an exception
                mmSocket.connect();
            } catch (IOException connectException) {
                // Unable to connect; close the socket and get out
                try {
                    mmSocket.close();
                } catch (IOException closeException) { }
                // Send the name of the disconnected device back to the UI Activity
                sendDeviceConnectionToActivity(deviceMAC, false);
                Log.d("Bluetoot connected -->", "NNNNNNNN" + connectException);
                return;
            }

            // Do work to manage the connection (in a separate thread)
            manageConnectedSocket(mmSocket, mmDevice);
//            mConnectedThread = new ConnectedThread(mmSocket);
//            mConnectedThread.start();
            Log.d("Bluetoot connected -->", mmDevice.getName());
        }

        /** Will cancel an in-progress connection, and close the socket */
        public void cancel() {
            try {
                mmSocket.close();
            } catch (IOException e) { }
        }
    }

the code for audio in thread 2 and 3 (this code is with a another class)

public void audioCreate() {
        // Audio track object
        track = new AudioTrack(AudioManager.STREAM_VOICE_CALL,
                16000, AudioFormat.CHANNEL_OUT_MONO,
                encoding, minSize, AudioTrack.MODE_STREAM);
        // Audio record object
        recorder = new AudioRecord(MediaRecorder.AudioSource.VOICE_COMMUNICATION, 16000,
                AudioFormat.CHANNEL_IN_MONO, encoding,
                bufferSize);
    }

public void initiateBluetoothConexion(BluetoothDevice deviceSelected) {
//        Toast.makeText(getApplicationContext(), "Service On", Toast.LENGTH_SHORT).show();
        deviceMAC = deviceSelected.getAddress();

        mBluetoothAdapter.cancelDiscovery();
        // Cancel any thread attempting to make a connection
        if (mConnectThread != null) {
            mConnectThread.cancel();
            mConnectThread = null;
        }
        mConnectThread = new ConnectThread(deviceSelected);
        mConnectThread.setPriority(10);
        mConnectThread.start();
    }

    public class ConnectThread extends Thread {
        private final BluetoothSocket mmSocket;
        private final BluetoothDevice mmDevice;

        public ConnectThread(BluetoothDevice device) {
            // As mmSocket is final, we use a temporary socket variable
            BluetoothSocket tmp = null;
            mmDevice = device;

            // Get a BluetoothSocket to connect with the given BluetoothDevice
            try {
                // MY_UUID is the app's UUID string, also used by the server code
                tmp = device.createRfcommSocketToServiceRecord(UUID);
            } catch (IOException e) { }
            mmSocket = tmp;
        }

        public void run() {
            android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_AUDIO);
            // Cancel discovery because it will slow down the connection
            mBluetoothAdapter.cancelDiscovery();

            try {
                // Connect the device through the socket. This will block
                // until it succeeds or throws an exception
                mmSocket.connect();
            } catch (IOException connectException) {
                // Unable to connect; close the socket and get out
                try {
                    mmSocket.close();
                } catch (IOException closeException) { }
                // Send the name of the disconnected device back to the UI Activity
                sendDeviceConnectionToActivity(deviceMAC, false);
                return;
            }

            // Do work to manage the connection (in a separate thread)
            manageConnectedSocket(mmSocket, mmDevice);
//            mConnectedThread = new ConnectedThread(mmSocket);
//            mConnectedThread.start();
            Log.d("Bluetoot connected -->", mmDevice.getName());
        }

        /** Will cancel an in-progress connection, and close the socket */
        public void cancel() {
            try {
                mmSocket.close();
            } catch (IOException e) { }
        }
    }

    private void manageConnectedSocket(BluetoothSocket mmSocket, BluetoothDevice mmDevice) {
        // Cancel the thread that completed the connection
//        if (mConnectThread != null) {
//            mConnectThread.cancel();
//            mConnectThread = null;
//        }
        // Cancel any thread currently running a connection
        if (mConnectedThread != null) {
            mConnectedThread.cancel();
            mConnectedThread = null;
        }
        // Start the thread to manage the connection and perform transmissions
        mConnectedThread = new ConnectedThread(mmSocket);
        mConnectedThread.setPriority(10);
        mConnectedThread.start();
        // Send the name of the connected device back to the UI Activity
        Log.d(TAG, "Connected to " + mmDevice.getName());
        sendDeviceConnectionToActivity(mmDevice.getAddress(), true);
//        setState(STATE_CONNECTED);
    }

    public class ConnectedThread extends Thread {
        private final BluetoothSocket mmSocket;
        private final InputStream mmInStream;
        private final OutputStream mmOutStream;
        private byte buffer[] = null;
        private byte playBuffer[] = null;
        private boolean intercomm = false;

        public ConnectedThread(BluetoothSocket socket) {
            mmSocket = socket;
            InputStream tmpIn = null;
            OutputStream tmpOut = null;

            // Get the input and output streams; using temp objects because
            // member streams are final.
            try {
                tmpIn = socket.getInputStream();
            } catch (IOException e) {
                Log.e(TAG, "Error occurred when creating input stream", e);
            }
            try {
                tmpOut = socket.getOutputStream();
            } catch (IOException e) {
                Log.e(TAG, "Error occurred when creating output stream", e);
            }

            mmInStream = tmpIn;
            mmOutStream = tmpOut;
            intercomm = true;
        }

        public void run() {
            android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO);
            playBuffer = new byte[minSize];
            // Playback received audio
            track.play();

            startRecording();

            // receive recording until an exception occurs.
            while (intercomm) {
                try {
                    if (mmInStream.available() == 0) {
                        //Do nothing
                    } else {
                        mmInStream.read(playBuffer);
                        track.write(playBuffer, 0, playBuffer.length);
                    }
                } catch (IOException e) {
                    Log.d("AUDIO", "Error when receiving recording");
                    sendDeviceConnectionToActivity(deviceMAC, false);
                    break;
                }
            }
        }

        // Record Audio
        public void startRecording() {
            Log.d("AUDIO", "Assigning recorder");
            buffer = new byte[bufferSize];

            // Start Recording
            recorder.startRecording();
            Log.d("startRecording", "passed");
            // Start a thread
            recordingThread = new Thread(new Runnable() {
                @Override
                public void run() {
                    android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO);
                    Log.d("startRecording", "sendRecording");
                    sendRecording();
                }
            }, "AudioRecorder Thread");
            recordingThread.setPriority(10);
            recordingThread.start();
        }
        // Method for sending Audio
        public void sendRecording() {
            // Infinite loop until microphone button is released
            while (intercomm) {
                try {
                    recorder.read(buffer, 0, bufferSize);
                    mmOutStream.write(buffer);
                } catch (IOException e) {
                    Log.d("AUDIO", "Error when sending recording");
                    sendErrorsToActivity("Error sending audio");
                }
            }
        }

        // Call this method from the main activity to shut down the connection.
        public void cancel() {
            intercomm = false;
            stopPlaying();
            stopRecording();
            destroyProcesses();
            try {
                mmSocket.close();
            } catch (IOException e) {
                Log.e(TAG, "Could not close the connect socket", e);
            }
        }

        // Stop playing and free up resources
        public void stopPlaying() {
            if (track != null) {
                track.stop();
                track.flush();
            }
        }

        // Stop Recording and free up resources
        public void stopRecording() {
            if (recorder != null) {
                recorder.stop();
            }
        }

        public void destroyProcesses() {
            //Release resources for audio objects
            track.release();
            recorder.release();
        }
    }


I tested the code in an octacore android oreo. However, when I did it in an phone sdk 23, it was worst.

1 Answers1

0

Your AudioTrack is starving because it is not receiving data quickly enough from the arduino. This is most likely due to increased network contention during the BT connection process.

You appear to be configuring your AudioTrack with the smallest possible play-buffer. On most devices, this is only a few ms of audio, so if the AudioTrack isn't fed more data every few ms, it will starve, and you will hear a glitch.

One solution is to increase the AudioTrack's buffer size (perhaps to around 8000 samples or more).

In addition, you have not checked the return value from mmInStream.read(), which means you may be trying to "play" a playBuffer that is only partially filled.

Altering the thread priorities, as you are, is unlikely to make a qualitative difference.

greeble31
  • 4,894
  • 2
  • 16
  • 30
  • Thank you for your anwer, it really makes sense. Do you have a book or online resource where I could find how to work correctly playing and recording audio? Also, how do I check the return value from mmInStream.read()? should I wait until that value become -1? Thank you – Javier Castellanos Cruz Sep 09 '19 at 16:15
  • Don't forget to accept this answer if you believe it is correct! As to your additional questions: Designing an app of this type is very complicated, as there are a number of edge cases that can arising while streaming. There is enough material there to make a college course on the subject. You can search for other answers on this site. You should also look into terms like [SIP](https://en.wikipedia.org/wiki/Session_Initiation_Protocol), and consider existing frameworks, like WebRTC. – greeble31 Sep 09 '19 at 17:18
  • About `read()-ing` in particular: A properly written app will always have one thread blocking on a socket `read()` call (unless you want to use [NIO](https://developer.android.com/reference/java/nio/package-summary), but I'm not sure if that's possible with BT). After all, new data can arrive at any time, and you can't be sure how much or how little you're going to get. You can't just call `read()` until it returns -1, b/c that will only happen when the socket disconnects. – greeble31 Sep 09 '19 at 17:22