How to show camera preview on other device over Bluetooth?

Question

I am trying to create a Bluetooth camera app where 2 device connect via Bluetooth and on click of start button on device 1 it should start the camera on device 2 and camera preview should be shown on both the device. When user click capture on device 1, device 2 should capture a photo.

Full project is here

I am using Bluetooth chat sample code, I am able to give start command but I am not getting how to show stream of preview data from the camera which is in byte[] and show a preview on the other device

I tried to convert byte[] to bitmap but bitmap is null.

Preview data on device 1

camera.setPreviewCallback(new Camera.PreviewCallback() {
            @Override
            public void onPreviewFrame(byte[] data, Camera camera) {

                  //Here I am writing data through connected channel
                  //tried to convert byte[] to bitmap here also but its showing null
             BitmapFactory.Options options = new BitmapFactory.Options();
            bitmap = BitmapFactory.decodeByteArray(image, 0, image.length, options); 
               //setting bitmap to imageview but bitmap itself is null


            }

On the other device it continuously reading data written by the device 1 and handler pass the read data in byte[] and this byte[] I am trying to show in either surfaceview or imageview but both are not working

BluetoothCameraFragment

public class BluetoothCameraFragment extends Fragment implements SurfaceHolder.Callback{


TextView testView;
Camera camera;
SurfaceView surfaceView;
SurfaceHolder surfaceHolder;
Camera.PictureCallback rawCallback;
Camera.ShutterCallback shutterCallback;
Camera.PictureCallback jpegCallback;
private final String tag = "tagg";

Button start, stop, capture;

private static final String TAG = "BluetoothCamera";


private static final int REQUEST_CONNECT_DEVICE_SECURE = 1;
private static final int REQUEST_CONNECT_DEVICE_INSECURE = 2;
private static final int REQUEST_ENABLE_BT = 3;


private ListView mConversationView;
private EditText mOutEditText;
private Button mSendButton;
private ImageView imageview;


private String mConnectedDeviceName = null;


private ArrayAdapter<String> mConversationArrayAdapter;


private StringBuffer mOutStringBuffer;


private BluetoothAdapter mBluetoothAdapter = null;


private BluetoothCameraManager mCameraService = null;
private boolean isCameraRunning = false;

@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setHasOptionsMenu(true);

    mBluetoothAdapter = BluetoothAdapter.getDefaultAdapter();


    if (mBluetoothAdapter == null) {
        FragmentActivity activity = getActivity();
        Toast.makeText(activity, "Bluetooth is not available", Toast.LENGTH_LONG).show();
        activity.finish();
    }
}


@Override
public void onStart() {
    super.onStart();


    if (!mBluetoothAdapter.isEnabled()) {
        Intent enableIntent = new Intent(BluetoothAdapter.ACTION_REQUEST_ENABLE);
        startActivityForResult(enableIntent, REQUEST_ENABLE_BT);

    } else if (mCameraService == null) {
        setup();
    }
}

@Override
public void onDestroy() {
    super.onDestroy();
    if (mCameraService != null) {
        mCameraService.stop();
    }
}

@Override
public void onResume() {
    super.onResume();




    if (mCameraService != null) {

        if (mCameraService.getState() == BluetoothCameraManager.STATE_NONE) {

            mCameraService.start();
        }
    }
}

@Override
public View onCreateView(LayoutInflater inflater, @Nullable ViewGroup container,
                         @Nullable Bundle savedInstanceState) {
    return inflater.inflate(R.layout.fragment_bluetooth_camera, container, false);
}

@Override
public void onViewCreated(View view, @Nullable Bundle savedInstanceState) {

    mSendButton = (Button) view.findViewById(R.id.button_send);
    stop = (Button) view.findViewById(R.id.stop);
    capture = (Button) view.findViewById(R.id.capture);
    imageview = (ImageView) view.findViewById(R.id.previewImage);
}


private void setup() {
    Log.d(TAG, "setup()");












    mSendButton.setOnClickListener(new Button.OnClickListener()
    {
        public void onClick(View arg0) {

            sendMessage("start-camera".getBytes());

        }
    });

    View view = getView();
    if (null != view) {
        stop = (Button)view.findViewById(R.id.stop);
        capture = (Button) view.findViewById(R.id.capture);
    }

    stop.setOnClickListener(new Button.OnClickListener()
    {
        public void onClick(View arg0) {
            sendMessage("stop-camera".getBytes());
        }
    });
    capture.setOnClickListener(new View.OnClickListener() {

        @Override
        public void onClick(View v) {

            sendMessage("take-picture".getBytes());
        }
    });

    surfaceView = (SurfaceView)view.findViewById(R.id.surfaceview);
    surfaceHolder = surfaceView.getHolder();
    surfaceHolder.addCallback(this);
    surfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    rawCallback = new Camera.PictureCallback() {
        public void onPictureTaken(byte[] data, Camera camera) {
            Log.d("Log", "onPictureTaken - raw");
        }
    };


    shutterCallback = new Camera.ShutterCallback() {
        public void onShutter() {
            Log.i("Log", "onShutter'd");
        }
    };
    jpegCallback = new Camera.PictureCallback() {
        public void onPictureTaken(byte[] data, Camera camera) {
            FileOutputStream outStream = null;
            try {
                outStream = new FileOutputStream(String.format(
                        "/sdcard/%d.jpg", System.currentTimeMillis()));
                outStream.write(data);
                outStream.close();
                Log.d("Log", "onPictureTaken - wrote bytes: " + data.length);
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            } finally {
            }
            Log.d("Log", "onPictureTaken - jpeg");
        }
    };



    mCameraService = new BluetoothCameraManager(getActivity(), mHandler);


    mOutStringBuffer = new StringBuffer("");
}


private void ensureDiscoverable() {
    if (mBluetoothAdapter.getScanMode() !=
            BluetoothAdapter.SCAN_MODE_CONNECTABLE_DISCOVERABLE) {
        Intent discoverableIntent = new Intent(BluetoothAdapter.ACTION_REQUEST_DISCOVERABLE);
        discoverableIntent.putExtra(BluetoothAdapter.EXTRA_DISCOVERABLE_DURATION, 300);
        startActivity(discoverableIntent);
    }
}


private void sendMessage(byte[] preview) {

    if (mCameraService.getState() != BluetoothCameraManager.STATE_CONNECTED) {
        Toast.makeText(getActivity(), R.string.not_connected, Toast.LENGTH_SHORT).show();
        return;
    }


    Log.d("sttatt","sending data");
        mCameraService.write(preview);



}


private TextView.OnEditorActionListener mWriteListener
        = new TextView.OnEditorActionListener() {
    public boolean onEditorAction(TextView view, int actionId, KeyEvent event) {

        if (actionId == EditorInfo.IME_NULL && event.getAction() == KeyEvent.ACTION_UP) {
            String message = view.getText().toString();

        }
        return true;
    }
};


private void setStatus(int resId) {
    FragmentActivity activity = getActivity();
    if (null == activity) {
        return;
    }
    final ActionBar actionBar = activity.getActionBar();
    if (null == actionBar) {
        return;
    }
    actionBar.setSubtitle(resId);
}


private void setStatus(CharSequence subTitle) {
    FragmentActivity activity = getActivity();
    if (null == activity) {
        return;
    }
    final ActionBar actionBar = activity.getActionBar();
    if (null == actionBar) {
        return;
    }
    actionBar.setSubtitle(subTitle);
}


private final Handler mHandler = new Handler() {
    @Override
    public void handleMessage(Message msg) {
        FragmentActivity activity = getActivity();
        switch (msg.what) {
            case Constants.MESSAGE_STATE_CHANGE:
                switch (msg.arg1) {
                    case BluetoothCameraManager.STATE_CONNECTED:
                        setStatus(getString(R.string.title_connected_to, mConnectedDeviceName));

                        break;
                    case BluetoothCameraManager.STATE_CONNECTING:
                        setStatus(R.string.title_connecting);
                        break;
                    case BluetoothCameraManager.STATE_LISTEN:
                    case BluetoothCameraManager.STATE_NONE:
                        setStatus(R.string.title_not_connected);
                        break;
                }
                break;
            case Constants.START_CAMERA_SERVICE:

                byte[] readBuf = (byte[]) msg.obj;
                String command  = new String(readBuf).toString();
                Log.d("cammy", "" + command);
                if(command.equals("start-camera")){
                    Log.d("cammy","Startcam");
                    start_camera();
                    Toast.makeText(getActivity(),"starting camera",Toast.LENGTH_LONG).show();

                }else if(command.equals("stop-camera")){
                    Log.d("cammy","Stopcam");
                    stop_camera();
                    Toast.makeText(getActivity(),"stopping camera",Toast.LENGTH_LONG).show();
                }else if(command.equals("take-picture")){
                    Log.d("cammy","takepic");
                    captureImage();
                    Toast.makeText(getActivity(),"Take picture",Toast.LENGTH_LONG).show();
                }else {
                    Log.d("cammy","No trigger");
                }



                break;
            case Constants.STOP_CAMERA:

                break;
            case Constants.TAKE_PICTURE:

                break;
            case Constants.MESSAGE_WRITE:

                mSendButton.setClickable(false);


                Log.d("sttatt","writing data");





                break;
            case Constants.MESSAGE_READ:








                break;
            case Constants.MESSAGE_DEVICE_NAME:

                mConnectedDeviceName = msg.getData().getString(Constants.DEVICE_NAME);
                if (null != activity) {
                    Toast.makeText(activity, "Connected to "
                            + mConnectedDeviceName, Toast.LENGTH_SHORT).show();
                }
                break;
            case Constants.MESSAGE_TOAST:
                if (null != activity) {
                    Toast.makeText(activity, msg.getData().getString(Constants.TOAST),
                            Toast.LENGTH_SHORT).show();
                }
                break;
        }
    }
};

public void onActivityResult(int requestCode, int resultCode, Intent data) {
    switch (requestCode) {
        case REQUEST_CONNECT_DEVICE_SECURE:

            if (resultCode == Activity.RESULT_OK) {
                connectDevice(data, true);
            }
            break;
        case REQUEST_CONNECT_DEVICE_INSECURE:

            if (resultCode == Activity.RESULT_OK) {
                connectDevice(data, false);
            }
            break;
        case REQUEST_ENABLE_BT:

            if (resultCode == Activity.RESULT_OK) {

                setup();
            } else {

                Log.d(TAG, "BT not enabled");
                Toast.makeText(getActivity(), R.string.bt_not_enabled_leaving,
                        Toast.LENGTH_SHORT).show();
                getActivity().finish();
            }
    }
}


private void connectDevice(Intent data, boolean secure) {

    String address = data.getExtras()
            .getString(DeviceListActivity.EXTRA_DEVICE_ADDRESS);

    BluetoothDevice device = mBluetoothAdapter.getRemoteDevice(address);

    mCameraService.connect(device, secure);
}

@Override
public void onCreateOptionsMenu(Menu menu, MenuInflater inflater) {
    inflater.inflate(R.menu.bluetooth_camera, menu);
}

@Override
public boolean onOptionsItemSelected(MenuItem item) {
    switch (item.getItemId()) {
        case R.id.secure_connect_scan: {

            Intent serverIntent = new Intent(getActivity(), DeviceListActivity.class);
            startActivityForResult(serverIntent, REQUEST_CONNECT_DEVICE_SECURE);
            return true;
        }
        case R.id.insecure_connect_scan: {

            Intent serverIntent = new Intent(getActivity(), DeviceListActivity.class);
            startActivityForResult(serverIntent, REQUEST_CONNECT_DEVICE_INSECURE);
            return true;
        }
        case R.id.discoverable: {

            ensureDiscoverable();
            return true;
        }
    }
    return false;
}


private void captureImage() {
    camera.takePicture(shutterCallback, rawCallback, jpegCallback);
}

private void start_camera(){
    try{
        camera = Camera.open();
        isCameraRunning=true;
    }catch(RuntimeException e){
        Log.e(tag, "init_camera: " + e);
        return;
    }
    Camera.Parameters param;
    param = camera.getParameters();

    param.setPreviewFrameRate(20);
    param.setPreviewSize(176, 144);
    camera.setParameters(param);
    try {
        camera.setPreviewDisplay(surfaceHolder);
        camera.setPreviewCallback(new Camera.PreviewCallback() {
            @Override
            public void onPreviewFrame(byte[] data, Camera camera) {




            }
        });
        camera.startPreview();

    } catch (Exception e) {
        Log.e(tag, "init_camera: " + e);
        return;
    }
}

private void stop_camera()
{
        camera.stopPreview();
        camera.release();
}

public void surfaceChanged(SurfaceHolder arg0, int arg1, int arg2, int arg3) {

}

public void surfaceCreated(SurfaceHolder holder) {

}

public void surfaceDestroyed(SurfaceHolder holder) {

}

}

BluetoothManager

 public class BluetoothCameraManager {

private static final String TAG = "rolf";

private static final String NAME_SECURE = "BluetoothCameraSecure";
private static final String NAME_INSECURE = "BluetoothCameraInsecure";


private static final UUID MY_UUID_SECURE =
        UUID.fromString("fa87c0d0-afac-11de-8a39-0800200c9a66");
private static final UUID MY_UUID_INSECURE =
        UUID.fromString("8ce255c0-200a-11e0-ac64-0800200c9a66");


private final BluetoothAdapter mAdapter;
private final Handler mHandler;
private AcceptThread mSecureAcceptThread;
private AcceptThread mInsecureAcceptThread;
private ConnectThread mConnectThread;
private ConnectedThread mConnectedThread;
private int mState;


public static final int STATE_NONE = 0;       
public static final int STATE_LISTEN = 1;     
public static final int STATE_CONNECTING = 2; 
public static final int STATE_CONNECTED = 3;  


public BluetoothCameraManager(Context context, Handler handler) {
    mAdapter = BluetoothAdapter.getDefaultAdapter();
    mState = STATE_NONE;
    mHandler = handler;
}


private synchronized void setState(int state) {
    Log.d(TAG, "setState() " + mState + " -> " + state);
    mState = state;


    mHandler.obtainMessage(Constants.MESSAGE_STATE_CHANGE, state, -1).sendToTarget();
}


public synchronized int getState() {
    return mState;
}


public synchronized void start() {
    Log.d(TAG, "start");


    if (mConnectThread != null) {
        mConnectThread.cancel();
        mConnectThread = null;
    }


    if (mConnectedThread != null) {
        mConnectedThread.cancel();
        mConnectedThread = null;
    }

    setState(STATE_LISTEN);


    if (mSecureAcceptThread == null) {
        mSecureAcceptThread = new AcceptThread(true);
        mSecureAcceptThread.start();
    }
    if (mInsecureAcceptThread == null) {
        mInsecureAcceptThread = new AcceptThread(false);
        mInsecureAcceptThread.start();
    }
}


public synchronized void connect(BluetoothDevice device, boolean secure) {
    Log.d(TAG, "connect to: " + device);


    if (mState == STATE_CONNECTING) {
        if (mConnectThread != null) {
            mConnectThread.cancel();
            mConnectThread = null;
        }
    }


    if (mConnectedThread != null) {
        mConnectedThread.cancel();
        mConnectedThread = null;
    }


    mConnectThread = new ConnectThread(device, secure);
    mConnectThread.start();
    setState(STATE_CONNECTING);
}


public synchronized void connected(BluetoothSocket socket, BluetoothDevice
        device, final String socketType) {
    Log.d(TAG, "connected, Socket Type:" + socketType);


    if (mConnectThread != null) {
        mConnectThread.cancel();
        mConnectThread = null;
    }


    if (mConnectedThread != null) {
        mConnectedThread.cancel();
        mConnectedThread = null;
    }


    if (mSecureAcceptThread != null) {
        mSecureAcceptThread.cancel();
        mSecureAcceptThread = null;
    }
    if (mInsecureAcceptThread != null) {
        mInsecureAcceptThread.cancel();
        mInsecureAcceptThread = null;
    }


    mConnectedThread = new ConnectedThread(socket, socketType);
    mConnectedThread.start();


    Message msg = mHandler.obtainMessage(Constants.MESSAGE_DEVICE_NAME);
    Bundle bundle = new Bundle();
    bundle.putString(Constants.DEVICE_NAME, device.getName());
    msg.setData(bundle);
    mHandler.sendMessage(msg);

    setState(STATE_CONNECTED);
}


public synchronized void stop() {
    Log.d(TAG, "stop");

    if (mConnectThread != null) {
        mConnectThread.cancel();
        mConnectThread = null;
    }

    if (mConnectedThread != null) {
        mConnectedThread.cancel();
        mConnectedThread = null;
    }

    if (mSecureAcceptThread != null) {
        mSecureAcceptThread.cancel();
        mSecureAcceptThread = null;
    }

    if (mInsecureAcceptThread != null) {
        mInsecureAcceptThread.cancel();
        mInsecureAcceptThread = null;
    }
    setState(STATE_NONE);
}


public void write(byte[] out) {

    ConnectedThread r;

    synchronized (this) {
        if (mState != STATE_CONNECTED) return;
        r = mConnectedThread;
    }

    r.write(out);
}


private void connectionFailed() {

    Message msg = mHandler.obtainMessage(Constants.MESSAGE_TOAST);
    Bundle bundle = new Bundle();
    bundle.putString(Constants.TOAST, "Unable to connect device");
    msg.setData(bundle);
    mHandler.sendMessage(msg);


    BluetoothCameraManager.this.start();
}


private void connectionLost() {

    Message msg = mHandler.obtainMessage(Constants.MESSAGE_TOAST);
    Bundle bundle = new Bundle();
    bundle.putString(Constants.TOAST, "Device connection was lost");
    msg.setData(bundle);
    mHandler.sendMessage(msg);

    BluetoothCameraManager.this.start();
}


private class AcceptThread extends Thread {

    private final BluetoothServerSocket mmServerSocket;
    private String mSocketType;

    public AcceptThread(boolean secure) {
        BluetoothServerSocket tmp = null;
        mSocketType = secure ? "Secure" : "Insecure";


        try {
            if (secure) {
                tmp = mAdapter.listenUsingRfcommWithServiceRecord(NAME_SECURE,
                        MY_UUID_SECURE);
            } else {
                tmp = mAdapter.listenUsingInsecureRfcommWithServiceRecord(
                        NAME_INSECURE, MY_UUID_INSECURE);
            }
        } catch (IOException e) {
            Log.e(TAG, "Socket Type: " + mSocketType + "listen() failed", e);
        }
        mmServerSocket = tmp;
    }

    public void run() {
        Log.d(TAG, "Accept thread: " + mSocketType +
                "BEGIN mAcceptThread" + this);
        setName("AcceptThread" + mSocketType);

        BluetoothSocket socket = null;


        while (mState != STATE_CONNECTED) {
            try {


                socket = mmServerSocket.accept();
            } catch (IOException e) {
                Log.e(TAG, "Socket Type: " + mSocketType + "accept() failed", e);
                break;
            }


            if (socket != null) {
                synchronized (BluetoothCameraManager.this) {
                    switch (mState) {
                        case STATE_LISTEN:
                        case STATE_CONNECTING:

                            connected(socket, socket.getRemoteDevice(),
                                    mSocketType);
                            break;
                        case STATE_NONE:
                        case STATE_CONNECTED:

                            try {
                                socket.close();
                            } catch (IOException e) {
                                Log.e(TAG, "Could not close unwanted socket", e);
                            }
                            break;
                    }
                }
            }
        }
        Log.i(TAG, "END mAcceptThread, socket Type: " + mSocketType);

    }

    public void cancel() {
        Log.d(TAG, "Socket Type" + mSocketType + "cancel " + this);
        try {
            mmServerSocket.close();
        } catch (IOException e) {
            Log.e(TAG, "Socket Type" + mSocketType + "close() of server failed", e);
        }
    }
}



private class ConnectThread extends Thread {
    private final BluetoothSocket mmSocket;
    private final BluetoothDevice mmDevice;
    private String mSocketType;

    public ConnectThread(BluetoothDevice device, boolean secure) {
        mmDevice = device;
        BluetoothSocket tmp = null;
        mSocketType = secure ? "Secure" : "Insecure";



        try {
            if (secure) {
                tmp = device.createRfcommSocketToServiceRecord(
                        MY_UUID_SECURE);
            } else {
                tmp = device.createInsecureRfcommSocketToServiceRecord(
                        MY_UUID_INSECURE);
            }
        } catch (IOException e) {
            Log.e(TAG, "Socket Type: " + mSocketType + "create() failed", e);
        }
        mmSocket = tmp;
    }

    public void run() {
        Log.i(TAG, "BEGIN mConnectThread SocketType:" + mSocketType);
        setName("ConnectThread" + mSocketType);


        mAdapter.cancelDiscovery();


        try {


            mmSocket.connect();
        } catch (IOException e) {

            try {
                mmSocket.close();
            } catch (IOException e2) {
                Log.e(TAG, "unable to close() " + mSocketType +
                        " socket during connection failure", e2);
            }
            connectionFailed();
            return;
        }


        synchronized (BluetoothCameraManager.this) {
            mConnectThread = null;
        }


        connected(mmSocket, mmDevice, mSocketType);
    }

    public void cancel() {
        try {
            mmSocket.close();
        } catch (IOException e) {
            Log.e(TAG, "close() of connect " + mSocketType + " socket failed", e);
        }
    }
}


private class ConnectedThread extends Thread {
    private final BluetoothSocket mmSocket;
    private final InputStream mmInStream;
    private final OutputStream mmOutStream;

    public ConnectedThread(BluetoothSocket socket, String socketType) {
        Log.d(TAG, "create ConnectedThread: " + socketType);
        mmSocket = socket;
        InputStream tmpIn = null;
        OutputStream tmpOut = null;


        try {
            tmpIn = socket.getInputStream();
            tmpOut = socket.getOutputStream();
        } catch (IOException e) {
            Log.e(TAG, "temp sockets not created", e);
        }

        mmInStream = tmpIn;
        mmOutStream = tmpOut;
    }

    public void run() {
        Log.d(TAG, "BEGIN mConnectedThread");
        byte[] buffer = new byte[1024];
        int bytes;


        while (true) {
            try {

                bytes = mmInStream.read(buffer);
                mHandler.obtainMessage(Constants.START_CAMERA_SERVICE, bytes, -1, buffer)
                        .sendToTarget();




                Log.d(TAG, "Reading");



            } catch (IOException e) {
                Log.e(TAG, "disconnected", e);
                connectionLost();

                BluetoothCameraManager.this.start();
                break;
            }
        }
    }


    public void write(byte[] buffer) {
        try {
            mmOutStream.write(buffer);


        } catch (IOException e) {
            Log.e(TAG, "Exception during write", e);
        }
    }

    public void cancel() {
        try {
            mmSocket.close();
        } catch (IOException e) {
            Log.e(TAG, "close() of connect socket failed", e);
        }
    }
}
}

I am afraid the bandwidth of bluetooth is not sufficient for video mirroring. But even over Wi-Fi, the raw pixels (or bitmaps) are too huge for transport. Have a look at the [libstreaming project](https://github.com/fyhertz/libstreaming) for a working library that can send video stream from an Android camera to another screen over IP. — Alex Cohn, Feb 02 '16 at 08:46
Preview frames are **not** bitmaps that is why you get `null` in your bitmap. They are YUV usually (/always maybe). http://stackoverflow.com/q/4768165/360211 — weston, Feb 02 '16 at 09:36
@weston Thanks is there any way I can set this byte[] data to other surfaceview? or I have to convert byte[] to bitmap and set it to imageview — Shivaraj Patil, Feb 02 '16 at 10:43
done something as a test project (works only for samsung devices) at my workplace using http://developer.samsung.com/galaxy#chord — Amit K. Saha, Feb 02 '16 at 10:57

score 3 · Answer 1 · answered May 06 '16 at 08:29

Usually bluetooth is very slow to send image frames, but you don't need to sends each and every frames. One problem you will face is the frame collision. Bluetooth socket sends your data(frames) as byte array. So while receiving the data from other end most of the times the bytes of previous frame and current frame will collide. So you need to make sure that is the first frame is received by the remote mobile and processed successfully. The frames created while the remote mobile is processing should not be stores in buffer to send later, Rather they should be ignored(Should not send). Some frames may miss, but you will work as live feed. Now the android mobile will create big frames of preview according to the camera resolution. So you need to send small size of frame only, on the remaining space of the display you can show camera controls.

I have posted an example project in my Github repo. Bluetooth Camera

How to show camera preview on other device over Bluetooth?

1 Answers1

Linked