Create JavaFX Image from PGM or TIFF as fast as possible

Question

I'm capturing images from a scanner device with java. The input format ist PGM or TIFF. I have to show up live results in the user interface. Actually I'm using ImageJ to read the source input stream as tiff, because ImageJ can also handle incomplete streams. After that the ImagePlus object is converted into a BufferedImage and finally into a JavaFX Image.

ImagePlus imagePlus = new Opener().openTiff(inputStream, "");
BufferedImage bufferedImage = imagePlus.getBufferedImage();
Image image = SwingFXUtils.toFXImage(bufferedImage, null);

This is very slow. I need a faster way to create the JavaFX Image from the PGM or TIFF stream. It seems that JavaFX has actually no support for this formats and I don't found a usefull library.

Any idea?

Edit #1

I've decided to optimze the image capturing in two steps. At first I need a better state control when updating the image in the ui. This is actually done and works fine. Now update requests are dropped, when the conversion thread is busy. The second step is to use a self implemented pnm reader (based on the suggested implementation) and update the image in my model incrementally... until the scan process is complete. This should reduce the required recources when loading an image from the device. I need to change some parts of my architecture to make this happen.

Thanks @ all for comments.

btw: java 8 lambdas are great :)

Edit #2

My plan doesn't work, because of JavaFX's thread test :(

Currently I have a WritableImage in my backend wich should be filled step by step with data. This image instance is set to an ObjectProperty that is finally bound to the ImageView. Since the WritableImage is connected to the ImageView it's impossible to fill it with data by using a PixelWriter. This causes an exception.

java.lang.IllegalStateException: Not on FX application thread; currentThread = pool-2-thread-1
    at com.sun.javafx.tk.Toolkit.checkFxUserThread(Toolkit.java:210) ~[jfxrt.jar:na]
    at com.sun.javafx.tk.quantum.QuantumToolkit.checkFxUserThread(QuantumToolkit.java:393) ~[jfxrt.jar:na]
    at javafx.scene.Scene.addToDirtyList(Scene.java:529) ~[jfxrt.jar:na]
    at javafx.scene.Node.addToSceneDirtyList(Node.java:417) ~[jfxrt.jar:na]
    at javafx.scene.Node.impl_markDirty(Node.java:408) ~[jfxrt.jar:na]
    at javafx.scene.Node.transformedBoundsChanged(Node.java:3789) ~[jfxrt.jar:na]
    at javafx.scene.Node.impl_geomChanged(Node.java:3753) ~[jfxrt.jar:na]
    at javafx.scene.image.ImageView.access$700(ImageView.java:141) ~[jfxrt.jar:na]
    at javafx.scene.image.ImageView$3.invalidated(ImageView.java:285) ~[jfxrt.jar:na]
    at javafx.beans.WeakInvalidationListener.invalidated(WeakInvalidationListener.java:83) ~[jfxrt.jar:na]
    at com.sun.javafx.binding.ExpressionHelper$SingleInvalidation.fireValueChangedEvent(ExpressionHelper.java:135) ~[jfxrt.jar:na]
    at com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(ExpressionHelper.java:80) ~[jfxrt.jar:na]
    at javafx.beans.property.ReadOnlyObjectPropertyBase.fireValueChangedEvent(ReadOnlyObjectPropertyBase.java:74) ~[jfxrt.jar:na]
    at javafx.scene.image.Image$ObjectPropertyImpl.fireValueChangedEvent(Image.java:568) ~[jfxrt.jar:na]
    at javafx.scene.image.Image.pixelsDirty(Image.java:542) ~[jfxrt.jar:na]
    at javafx.scene.image.WritableImage$2.setArgb(WritableImage.java:170) ~[jfxrt.jar:na]
    at javafx.scene.image.WritableImage$2.setColor(WritableImage.java:179) ~[jfxrt.jar:na]

My workaround is to create a copy of the image, but I don't like this solution. Maybe it's possible to prevent the automatic change notification and do this manually?

Answer 1

As an experiment, and to learn some JavaFX, I decided to see for myself how hard it would be to implement what I suggested in the comment above... :-)

The PGM reading is adapted from my PNM ImageIO plugin, and it seems to work okay. Read times is reported to be around 70-90 ms for my 640x480 sample images (feel free to send me some more samples if you have!).

An uncompressed TIFF should be readable in roughly the same time, although the TIFF IFD structure is more complex to parse than the very simple PGM header. TIFF compression will add some decompression overhead, depending on compression type and settings.

import java.io.DataInputStream;
import java.io.IOException;

import javax.imageio.IIOException;

import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.image.PixelWriter;
import javafx.scene.image.WritableImage;
import javafx.scene.paint.Color;
import javafx.stage.Stage;

public class PGMTest extends Application {

    public static void main(String[] args) {
        launch(args);
    }

    @Override
    public void start(Stage primaryStage) throws IOException {
        Label root = new Label();
        Image image;

        long start = System.currentTimeMillis();
        DataInputStream input = new DataInputStream(getClass().getResourceAsStream("/house.l.pgm"));
        try {
            image = readImage(input);
        } finally {
            input.close();
        }
        System.out.printf("Read image (%f x %f) in: %d ms\n", image.getWidth(), image.getHeight(), System.currentTimeMillis() - start);

        root.setGraphic(new ImageView(image));
        Scene scene = new Scene(root);
        primaryStage.setScene(scene);
        primaryStage.show();
    }

    private Image readImage(final DataInputStream input) throws IOException {
        // First parse PGM header
        PNMHeader header = PNMHeader.parse(input);

        WritableImage image = new WritableImage(header.getWidth(), header.getHeight());
        PixelWriter pixelWriter = image.getPixelWriter();

        int maxSample = header.getMaxSample(); // Needed for normalization

//        PixelFormat<ByteBuffer> gray = PixelFormat.createByteIndexedInstance(createGrayColorMap());

        byte[] rowBuffer = new byte[header.getWidth()];
        for (int y = 0; y < header.getHeight(); y++) {
            input.readFully(rowBuffer); // Read one row

//            normalize(rowBuffer, maxSample);
//            pixelWriter.setPixels(0, y, width, 1, gray, rowBuffer, 0, width); // Gives weird NPE for me...

            // As I can't get setPixels to work, we'll set pixels directly
            // Performance is probably worse than setPixels, but it seems "ok"-ish
            for (int x = 0; x < rowBuffer.length; x++) {
                int gray = (rowBuffer[x] & 0xff) * 255 / maxSample; // Normalize [0...255]
                pixelWriter.setArgb(x, y, 0xff000000 | gray << 16 | gray << 8 | gray);
            }
        }

        return image;
    }

    private int[] createGrayColorMap() {
        int[] colors = new int[256];
        for (int i = 0; i < colors.length; i++) {
            colors[i] = 0xff000000 | i << 16 | i << 8 | i;
        }
        return colors;
    }

    /**
     * Simplified version of my PNMHeader parser
     */
    private static class PNMHeader {
        public static final int PGM = 'P' << 8 | '5';

        private final int width;
        private final int height;
        private final int maxSample;

        private PNMHeader(final int width, final int height, final int maxSample) {
            this.width = width;
            this.height = height;
            this.maxSample = maxSample;
        }

        public int getWidth() {
            return width;
        }

        public int getHeight() {
            return height;
        }

        public int getMaxSample() {
            return maxSample;
        }

        public static PNMHeader parse(final DataInputStream input) throws IOException {
            short type = input.readShort();

            if (type != PGM) {
                throw new IIOException(String.format("Only PGM binay (P5) supported for now: %04x", type));
            }

            int width = 0;
            int height = 0;
            int maxSample = 0;

            while (width == 0 || height == 0 || maxSample == 0) {
                String line = input.readLine(); // For PGM I guess this is ok...

                if (line == null) {
                    throw new IIOException("Unexpeced end of stream");
                }

                if (line.indexOf('#') >= 0) {
                    // Skip comment
                    continue;
                }

                line = line.trim();

                if (!line.isEmpty()) {
                    // We have tokens...
                    String[] tokens = line.split("\\s");
                    for (String token : tokens) {
                        if (width == 0) {
                            width = Integer.parseInt(token);
                        } else if (height == 0) {
                            height = Integer.parseInt(token);
                        } else if (maxSample == 0) {
                            maxSample = Integer.parseInt(token);
                        } else {
                            throw new IIOException("Unknown PBM token: " + token);
                        }
                    }
                }
            }

            return new PNMHeader(width, height, maxSample);
        }
    }
}

I should probably add that I wrote, compiled and ran the above code on Java 7, using JavaFX 2.2.

---

Update: Using a predefined PixelFormat I was able to use PixelWriter.setPixels and thus further reduce read times to 45-60 ms for the same 640x480 sample images. Here's a new version of readImage (the code is otherwise the same):

private Image readImage(final DataInputStream input) throws IOException {
    // First parse PGM header
    PNMHeader header = PNMHeader.parse(input);

    int width = header.getWidth();
    int height = header.getHeight();
    WritableImage image = new WritableImage(width, height);
    PixelWriter pixelWriter = image.getPixelWriter();

    int maxSample = header.getMaxSample(); // Needed to normalize

    PixelFormat<ByteBuffer> format = PixelFormat.getByteRgbInstance();

    byte[] rowBuffer = new byte[width * 3]; // * 3 to hold RGB 
    for (int y = 0; y < height; y++) {
        input.readFully(rowBuffer, 0, width); // Read one row

        // Expand gray to RGB triplets
        for (int i = width - 1; i > 0; i--) {
            byte gray = (byte) ((rowBuffer[i] & 0xff) * 255 / maxSample); // Normalize [0...255];
            rowBuffer[i * 3    ] = gray;
            rowBuffer[i * 3 + 1] = gray;
            rowBuffer[i * 3 + 2] = gray;
        }

        pixelWriter.setPixels(0, y, width, 1, format, rowBuffer, 0, width * 3);
    }

    return image;
}

Answer 2

Download jai_imageio.jar and include it in your project. Code to convert tiff images into fx readable images is below:

String pathToImage = "D:\\ABC.TIF";
ImageInputStream is;
try {
is = ImageIO.createImageInputStream(new File(pathToImage));  //read tiff using imageIO (JAI component)
if (is == null || is.length() == 0) {
    System.out.println("Image is null");
}

Iterator<ImageReader> iterator = ImageIO.getImageReaders(is);
if (iterator == null || !iterator.hasNext()) {
    throw new IOException("Image file format not supported by ImageIO: " + pathToImage);
}
ImageReader reader = (ImageReader) iterator.next();
reader.setInput(is);
int nbPages = reader.getNumImages(true);
BufferedImage bf = reader.read(0);   //1st page of tiff file
BufferedImage bf1 = reader.read(1);  //2nd page of tiff file
WritableImage wr = null;
WritableImage wr1 = null;
if (bf != null) {
    wr= SwingFXUtils.toFXImage(bf, null);   //convert bufferedImage (awt) into Writable Image(fx)
}
if (bf != null) {
    wr1= SwingFXUtils.toFXImage(bf1, null);  //convert bufferedImage (awt) into Writable Image(fx)
}
img_view1.setImage(wr);
img_view2.setImage(wr1);

} catch (FileNotFoundException ex) {
        Logger.getLogger(Image_WindowController.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
        Logger.getLogger(Image_WindowController.class.getName()).log(Level.SEVERE, null, ex);
}

This is my first answer on Stack Overflow. Hope it helps!