0

So I'm reading JFIF (JPEG) data from a file, as an exercise (I know there are libraries out there that already do this, I'm not looking for those). I've already got the image file size, color depth, and dimensions. However, I'm not too sure how to get the actual image data. I've looked at the data in a hex editor, and comparing that against the actual image leads me nowhere. If anyone has a good resource to start on this (I know it's probably an arduous and enlightening process, but that's why I'm doing it), that would be awesome.

My code so far, just for context:

// check header data, assign header data to important fields

        // Start Of Image (SOI) must be FFD8 and the next marker must be FF
        if(!(this.data[0] == (byte) 0xFF && this.data[1] == (byte) 0xD8
                && this.data[2] == (byte) 0xFF))
            this.isValid = false;

        // check if file is not valid
        if(!isValid) 
            log.log(Level.SEVERE, 
                    String.format("ERROR: File %s is not registered as a JFIF!\n", this.filename), 
                    new IllegalArgumentException());

        // If the next values are correct, then the data stream starts at SOI
        // If not, the data stream is raw
        this.isRawDataStream = !(this.data[3] == (byte) 0xE0
                && this.data[6]  == (byte) 0x4A
                && this.data[7]  == (byte) 0x46
                && this.data[8]  == (byte) 0x49
                && this.data[9]  == (byte) 0x46
                && this.data[10] == (byte) 0x00);

        // Read until SOF0 marker (0xC0)
        int i = 11;
        while(this.data[i] != (byte) 0xC0) {
            i++;
        }
        System.out.println("SOF0 marker at offset " + i);

        // Skip two bytes, next byte is the color depth
        this.colorDepth = this.data[i+3];

        // Next two bytes are the image height
        String h = String.format("%02X", this.data[i+4]) + String.format("%02X", this.data[i+5]);
        this.height = hexStringToInt(h);
        System.out.println("Height: " + this.height);

        // Next two bytes are the image width
        String w = String.format("%02X", this.data[i+6]) + String.format("%02X", this.data[i+7]); 
        this.width = hexStringToInt(w);
        System.out.println("Width: " + this.width);

        System.out.println("Color depth: " + this.colorDepth);
        // load pixels into an image
        this.image = new BufferedImage(this.width,
                                       this.height, 
                                       BufferedImage.TYPE_INT_RGB);

Then, I need to get each pixel and send it to the image. How would I get each pixel and its respective RGB data?

Tetramputechture
  • 2,911
  • 2
  • 33
  • 48
  • There is a specification you can read to see all the data needed for a jfif: http://www.jpeg.org/public/jfif.pdf – Obicere Oct 09 '14 at 15:03
  • 1
    Ah, so, according to reading, I have to get the RGB data like so? Y = 0.299 R + 0.587 G + 0.114 B Cb = - 0.1687 R - 0.3313 G + 0.5 B + 128 Cr = 0.5 R - 0.4187 G - 0.0813 B + 128 @Obicere – Tetramputechture Oct 09 '14 at 15:14
  • follow the specification. I haven't read over it, but there might be more to it than just that. But it will have all the required information needed to parse the image. – Obicere Oct 09 '14 at 15:18

1 Answers1

4

What you are trying to do is not a simple afternoon project. This book explains the process: There is A LOT of code between JPEG compressed data and pixel values.

http://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434/ref=pd_bxgy_b_img_y

First of all, you have to deal with two separate but related compression methods: Sequential and progressive.

As you read the bit data, you have to

  1. Huffman decode
  2. Run length decode
  3. Inverse Quantization
  4. List item
  5. Inverse Discrete Cosine Transform
  6. Up sample
  7. YCbCr to RGB convert

That's in the simple case of sequential.

You are not going to get all of those steps explained on this forum.

I also recommend

http://www.amazon.com/dp/1558514341/ref=rdr_ext_tmb

user3344003
  • 20,574
  • 3
  • 26
  • 62
  • 3
    This is not to discourage you but rather to set your expectations about the scope of the project. When I did what you are going many years ago, people sad not to even try. Just use a library. Then you don't learn anything and JPEG remains a black art. – user3344003 Oct 09 '14 at 16:21
  • 1
    Exactly! I'm glad somebody gets it! Also, I'm probably going to extensively read this: http://www.w3.org/Graphics/JPEG/itu-t81.pdf As it would probably guide me through the whole thing. – Tetramputechture Oct 09 '14 at 16:22
  • Kinda, sorta. The JPEG standard is not implementable. It lacks key details. It also includes so many ways to to things that it would be impossible to implement. Only sequential and progressive JPEG is commonly used (lossless is used in some 12-bit medical applications). You can ignore large parts of the standard. If I remember, the details on huffman coding are quite good. After learning to read all the different blocks, huffman coding is a good place to start next. That's Annex C. Ignore anything related to Hierarchical and arithmetic coding. – user3344003 Oct 09 '14 at 16:35
  • So I've already read all the blocks I need to (I think, I've read the SOI: color depth, image height/width, and number of components), so I can just start learning how to decode the huffman codes from the file? @user3344003 – Tetramputechture Oct 09 '14 at 16:42
  • 1
    You need the DHT and DQT blocks for a full decode. DHT is the huffman table. Annex C explains how to take the data in the DHT block and build a huffman table from that. Book books above show how to huffman decode. – user3344003 Oct 09 '14 at 16:48
  • I purchased the data compression book you linked! Thank you for all the help, I really needed it! @user3344003 – Tetramputechture Oct 09 '14 at 16:59
  • Hey, I've come pretty far (and that first book you linked helped quite a bit), but I still have some questions, can I chat with you? @user3344003 – Tetramputechture Oct 13 '14 at 23:59
  • Ask away. Also, there is some relatively easy to follow source code at http://www.colosseumbuilders.com/sourcecode/ – user3344003 Oct 14 '14 at 01:55
  • (I'm going to comment until it lets me start a chat.) So, I've got all the huffman table data, and organized my code much better than how it originally was. I have two arrays, huffCodeCounts[][], which holds the code counts for a table, and huffTables[][], which hold the actual tables. Each array has been filled with the correct data. However, I am a bit confused on what to do next; I know you're supposed to decode the tables, but **what does that mean?** – Tetramputechture Oct 14 '14 at 01:59
  • I've also made a function for the IDCT, so I've got that out of the way too. – Tetramputechture Oct 14 '14 at 02:49
  • I'm just having a hard time following the book section on the decoding process, and I'm stuck. I really need to get over this hump. – Tetramputechture Oct 14 '14 at 14:42
  • 1
    You should just use sequential jpeg images for now. Start by calculating the number MCUs needed for each component. That comes from the image size and sampling factors for the components. Next you're going to need to do huffman decoding. You need to be able to read the DHT markers and be able to build the table. If you have the counts, the standard tells you how to build the table from that. Counts gives the length of the huffman code for a value. The table will contain a bit sequence for each value. You're going to need code to read the compressed data stream in bits. – user3344003 Oct 14 '14 at 18:23
  • That's what I'm starting with. So now, I've got the number of MCUs needed for each component (I'm actually making progress! :D), and I've started the procedure to decode MCUs within a scan. However, when the book reads "Decode Data Unit" within the function, what does that mean? What do I pass to a theoretical function decode()? @user3344003 – Tetramputechture Oct 15 '14 at 03:28
  • And I've also figured how large each data unit is, so my only problem is figuring out how to go about decoding each data unit. I know it comprises several functions, but I don't know how to start. – Tetramputechture Oct 15 '14 at 14:20
  • I have the counts and the values from the DHT markers, and have all the value tables with their corresponding index/types. I've read the standard and the book, but I still don't know how to make tables from this data? I see the decoding function in the book, but I don't understand the global variables VALUES[], MINCODE[], MAXCODE[], and FIRSTDATA[]. @user3344003 – Tetramputechture Oct 15 '14 at 20:26
  • 1
    Mincode and Maxcode are what you use to decode. MinCode [I] is the smallest Huffman Code of length I. MaxCode [i] is the largest huffman code of length I. – user3344003 Oct 16 '14 at 03:40
  • How do you get a huffman code from counts and values, though? And what are Values[] and Firstdata[]? I'm sorry I'm bombarding you with questions, I have scoured everywhere for good source code that matches my project structure and I haven't found any that isn't in C or matlab. I just really need someone to talk to and walk me through this and tell me the steps I need to take. @user3344003 – Tetramputechture Oct 16 '14 at 03:43
  • Okay, I finally figured out how to get the actual code tables. I'm having a little trouble making my CountsToLengths function (I know LENGTHCOUNTS is the counts, but why is it [1..MAXIMUMLENGTH]? I thought counts just had 16 bytes?) I don't understand that, but the rest of it is fine. @user3344003 – Tetramputechture Oct 16 '14 at 14:53
  • The codes can be up to 16 bits long. So its [1 .. 16]. Sadly, C/C++ do not allow array indices to start at 1. – user3344003 Oct 16 '14 at 15:23
  • So, say my counts are `0 2 1 3 3 2 4 3 5 5 4 4 0 0 1 125`. When I run my function countsToLengths (from the book), I get an out of bounds exception. @user3344003 – Tetramputechture Oct 16 '14 at 15:34
  • Did you check the errata? http://www.colosseumbuilders.com/books/imageformats.html – user3344003 Oct 16 '14 at 17:19
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/63180/discussion-between-tetramputechture-and-user3344003). – Tetramputechture Oct 16 '14 at 17:28