I am using the CommonsWare CWAC-Camera library (https://github.com/commonsguy/cwac-camera) to wrap the native Android Camera API.
I would like to allow the user to select a point on the preview image before taking a picture, then (accurately) pick out that same point on the final image after they have taken the picture. While this works approximately, the markup on the final image is typically off-target by about 100-200 pixels.
To start with, I am capturing the user's original selected point by setting an OnTouchListener on the CameraView. I am then storing the X and Y coordinates from the MotionEvent. Just to help with visual verification, I am also immediately drawing a circle at the coordinates the user selected, before they take the picture.
When the user takes the picture, I have subclassed from SimpleCameraHost to override saveImage, in order to draw a corresponding circle on the final image before it is written to storage. In order to calculate where to draw the circle on the final image, I am doing the following:
- Create a Bitmap the same size and shape as the TextureView that was used to draw the preview
- Draw a circle on that Bitmap at the same coordinates as was drawn on-screen
- Scale the markup-Bitmap up to the final image size (using Matrix.ScaleToFit.Center) and overlay it on top of the actual final photo image
This is not sufficiently accurate, however - at least, not on the Nexus 7 (2013) that I am testing with. I am not sure what I am missing in the placement calculations for adding the markup to the final image.
A simplified example to demonstrate the problem follows:
MainActivity.java:
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
mCameraContainer = findViewById(R.id.camera_container);
mCanvasView = (CanvasView)findViewById(R.id.canvas_view);
FragmentManager fmgr = getFragmentManager();
FragmentTransaction ft = fmgr.beginTransaction();
mCameraFragment = (CameraFragment)fmgr.findFragmentByTag(CAMERA_FRAGMENT_TAG);
if (null == mCameraFragment) {
mCameraFragment = new CameraFragment();
ft.add(R.id.camera_container, mCameraFragment, CAMERA_FRAGMENT_TAG);
}
ft.commit();
if (null == mCameraHost) {
mCameraHost = new MyCameraHost(this);
}
mCameraFragment.setHost(mCameraHost);
ViewTreeObserver ccObserver = mCameraContainer.getViewTreeObserver();
ccObserver.addOnGlobalLayoutListener(new OnGlobalLayoutListener() {
@Override
public void onGlobalLayout() {
CameraView cv = (CameraView)mCameraFragment.getView();
View previewWidget = ((ViewGroup)cv).getChildAt(0);
mCameraHost.setPreviewSize(previewWidget.getWidth(), previewWidget.getHeight());
cv.setOnTouchListener(new OnTouchListener() {
@Override
public boolean onTouch(View v, MotionEvent e) {
mCanvasView.setPoint(e.getX(), e.getY());
mCameraHost.setPoint(e.getX(), e.getY());
return true;
}
});
mCameraContainer.getViewTreeObserver().removeOnGlobalLayoutListener(this);
}
});
Button photoButton = (Button)findViewById(R.id.takePhotoButton);
photoButton.setOnClickListener(new OnClickListener() {
@Override
public void onClick(View v) {
mCameraFragment.takePicture();
}
});
}
MyCameraHost.java:
@Override
public void saveImage(PictureTransaction xact, byte[] image) {
// decode the final image as a Bitmap
BitmapFactory.Options opt = new BitmapFactory.Options();
opt.inMutable = true;
Bitmap imageBitmap = BitmapFactory.decodeByteArray(image, 0, image.length, opt);
// draw a blank Bitmap with just the markup, using a canvas size equivalent to the preview view
Bitmap markerBitmap = Bitmap.createBitmap(previewViewWidth, previewViewHeight, Bitmap.Config.ARGB_8888);
Canvas c = new Canvas(markerBitmap);
Paint p = new Paint();
p.setStrokeWidth(10);
p.setStyle(Style.FILL);
p.setColor(Color.BLUE);
c.drawCircle(previewMarkerX, previewMarkerY, 20, p);
// scale the markup bitmap up to final-image size using Matrix.ScaleToFit.CENTER
Matrix m = new Matrix();
m.setRectToRect(new RectF(0, 0, previewViewWidth, previewViewHeight), new RectF(0, 0, imageBitmap.getWidth(), imageBitmap.getHeight()), Matrix.ScaleToFit.CENTER);
// overlay the scaled marker Bitmap onto the image
Canvas imageCanvas = new Canvas(imageBitmap);
imageCanvas.drawBitmap(markerBitmap, m, null);
// save the combined image
ByteArrayOutputStream bos = new ByteArrayOutputStream();
imageBitmap.compress(CompressFormat.JPEG, 70, bos);
byte[] image2 = bos.toByteArray();
super.saveImage(xact, image2);
}
layout XML:
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity" >
<FrameLayout
android:id="@+id/camera_container"
android:layout_width="match_parent"
android:layout_height="match_parent" />
<com.example.cwaccameratouchpoint.CanvasView
android:id="@+id/canvas_view"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignLeft="@id/camera_container"
android:layout_alignRight="@id/camera_container"
android:layout_alignTop="@id/camera_container"
android:layout_alignBottom="@id/camera_container" />
<Button
android:id="@+id/takePhotoButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignParentTop="true"
android:layout_alignParentRight="true"
android:text="go" />
</RelativeLayout>
As per comments from CommonsWare, I updated saveImage below to include the full-bleed offset and to spell out the transform calculations. It is an improvement, but still not as accurate as it should be:
// decode the final image as a Bitmap
BitmapFactory.Options opt = new BitmapFactory.Options();
opt.inMutable = true;
Bitmap imageBitmap = BitmapFactory.decodeByteArray(image, 0, image.length, opt);
int finalImageWidth = imageBitmap.getWidth();
int finalImageHeight = imageBitmap.getHeight();
// calculate selected point x and y values as applied to full-size final image
// apply full-bleed-based offset, where xoffset and yoffset represent the
// offset of the TextureView within its parent CameraView
int bleedAdjustedX = (int)previewMarkerX - xoffset;
int bleedAdjustedY = (int)previewMarkerY - yoffset;
// calculate offset for change in aspect ratio
// for now, assume portrait orientation only
double finalImageAspectRatio = (double)finalImageHeight / (double)finalImageWidth;
double aspectAdjustedWidth = (double)previewViewHeight / finalImageAspectRatio;
double aspectXOffset = (aspectAdjustedWidth - previewViewWidth) / 2;
int aspectAdjustedX = bleedAdjustedX + (int)aspectXOffset;
// scale adjusted coordinates for full-size image
double normalizedAdjustedX = (double)aspectAdjustedX / aspectAdjustedWidth;
double normalizedAdjustedY = (double)bleedAdjustedY / (double)previewViewHeight;
double scaledX = normalizedAdjustedX * (double)finalImageWidth;
double scaledY = normalizedAdjustedY * (double)finalImageHeight;
// draw markup on final image
Canvas c = new Canvas(imageBitmap);
Paint p = new Paint();
p.setStrokeWidth(10);
p.setStyle(Style.FILL);
p.setColor(Color.BLUE);
c.drawCircle((float)scaledX, (float)scaledY, 20, p);
// save marked-up image
ByteArrayOutputStream bos = new ByteArrayOutputStream();
imageBitmap.compress(CompressFormat.JPEG, 70, bos);
byte[] image2 = bos.toByteArray();
super.saveImage(xact, image2);
