ClipBoard What is an Object Descriptor type and how can i write it?

Question

So I made a small application that basicaly draw a whatever image is in the ClipBoard(memory) and trys to draw it.

This is a sample of the code:

private EventHandler<KeyEvent> copyPasteEvent =  new EventHandler() {
    final KeyCombination ctrl_V = new KeyCodeCombination(KeyCode.V, KeyCombination.CONTROL_DOWN);
    @Override
    public void handle(Event event) {
        if (ctrl_V.match((KeyEvent) event)) {

            System.out.println("Ctrl+V pressed");
            Clipboard clipboard = Clipboard.getSystemClipboard();

            System.out.println(clipboard.getContentTypes());

            //Change canvas size if necessary to allow space for the image to fit
            Image copiedImage = clipboard.getImage();
            if (copiedImage.getHeight()>canvas.getHeight()){
                canvas.setHeight(copiedImage.getHeight());
            }
            if (copiedImage.getWidth()>canvas.getWidth()){
                canvas.setWidth(copiedImage.getWidth());
            }

            gc.drawImage(clipboard.getImage(), 0,0);
        }
    }
};

This is the image that was drawn and the correspecting data type: A print from my screen.

A image from the internet.

However when i copy and paste a direct raw image from paint...

Answer 1

Object Descriptor is an OLE format from Microsoft.

This is why when you copy an image from a Microsoft application, you get these descriptors from Clipboard.getSystemClipboard().getContentTypes():

[[application/x-java-rawimage], [Object Descriptor]]

As for getting the image out of the clipboard... let's try two possible ways to do it: AWT and JavaFX.

AWT

Let's use the awt toolkit to get the system clipboard, and in case we have an image on it, retrieve a BufferedImage. Then we can convert it easily to a JavaFX Image and place it in an ImageView:

try {
    DataFlavor[] availableDataFlavors = Toolkit.getDefaultToolkit().
        getSystemClipboard().getAvailableDataFlavors();

    for (DataFlavor f : availableDataFlavors) {
        System.out.println("AWT Flavor: " + f);
        if (f.equals(DataFlavor.imageFlavor)) {

            BufferedImage data = (BufferedImage) Toolkit.getDefaultToolkit().getSystemClipboard().getData(DataFlavor.imageFlavor);
            System.out.println("data " + data);

            // Convert to JavaFX:
            WritableImage img = new WritableImage(data.getWidth(), data.getHeight());
            SwingFXUtils.toFXImage((BufferedImage) data, img);
            imageView.setImage(img);
        }
    }
} catch (UnsupportedFlavorException | IOException ex) {
    System.out.println("Error " + ex);
}

It prints:

AWT Flavor: java.awt.datatransfer.DataFlavor[mimetype=image/x-java-image;representationclass=java.awt.Image]

data BufferedImage@3e4eca95: type = 1 DirectColorModel: rmask=ff0000 gmask=ff00 bmask=ff amask=0 IntegerInterleavedRaster: width = 350 height = 364 #Bands = 3 xOff = 0 yOff = 0 dataOffset[0] 0

and displays your image:

This part was based in this answer.

JavaFX

Why didn't we try it with JavaFX in the first place? Well, we could have tried directly:

Image content = (Image) Clipboard.getSystemClipboard().getContent(DataFormat.IMAGE);
imageView.setImage(content);

and you will get a valid image, but when adding it to an ImageView, it will be blank as you already noticed, or with invalid colors.

So how can we get a valid image? If you check the BufferedImage above, it shows type = 1, which means BufferedImage.TYPE_INT_RGB = 1;, in other words, it is an image with 8-bit RGB color components packed into integer pixels, without alpha component.

My guess is that JavaFX implementation for Windows doesn't process correctly this image format, as it probably expects a RGBA format. You can check here how the image is extracted. And if you want to dive into the native implementation, check the native-glass/win/GlassClipboard.cpp code.

So we can try to do it with a PixelReader. Let's read the image and return a byte array:

private byte[] imageToData(Image image) {
    int width = (int) image.getWidth();
    int height = (int) image.getHeight();

    byte[] data = new byte[width * height * 3];

    int i = 0;
    for (int y = 0; y < height; y++) {
        for (int x = 0; x < width; x++) {
            int argb = image.getPixelReader().getArgb(x, y);
            int r = (argb >> 16) & 0xFF;
            int g = (argb >>  8) & 0xFF;
            int b =  argb        & 0xFF;
            data[i++] = (byte) r;
            data[i++] = (byte) g;
            data[i++] = (byte) b;
        }
      }
    return data;
}

Now, all we need to do is use this byte array to write a new image and set it to the ImageView:

Image content = (Image) Clipboard.getSystemClipboard().getContent(DataFormat.IMAGE);
byte[] data = imageToData(content);

WritableImage writableImage = new WritableImage((int) content.getWidth(), (int) content.getHeight());
PixelWriter pixelWriter = writableImage.getPixelWriter();
pixelWriter.setPixels(0, 0, (int) content.getWidth(), (int) content.getHeight(), 
        PixelFormat.getByteRgbInstance(), data, 0, (int) content.getWidth() * 3);
imageView.setImage(writableImage);

And now you will get the same result, but only using JavaFX: