Java - Compress large string

Question

In my Java application I get from some calculations a really long string (from really long string I mean of around 600000 characters or so). But I need to send this string to a client to process it, and for this reason I need the compressed string to be of maximum 1000 characters.

I have tried using GZIPOutputStream and Inflater and Deflater classes, and in the best case I got an output string of 300000 characters, which is great compression, but in my case it's not enough.

I have also tried compressing the string n times, but the ouput was larger than the previous one, so only one-time compressing was successful.

So, what do you suggest me to try?

Thank you.

Are you mathematically sure that there actually is a theoretical way to compress your data even more than that? — Florian Schaetz, Mar 13 '16 at 14:55
If you are limited to 1000 bytes then you are limited in how much information you can send at once. You have to consider other options as compression can only remove so much. Can you send the data in portions? — Peter Lawrey, Mar 13 '16 at 14:57
I could send it in portions, but only 2 or 3, due to client requirements — Ernestina Juan, Mar 13 '16 at 14:58
In that case, your client has chosen a limitation of ~3KB after compression and you can't send them more data than that. Either you get the client to agree to a high limit or you send less data. — Peter Lawrey, Mar 13 '16 at 14:59

score 0 · Accepted Answer · answered Mar 14 '16 at 13:43

I agree with @Peter Lawrey that, strictly with those requirements, it might be impossible to deliver such a big message to the client.

Anyway, I still suggest three possible solutions, depending on how flexible your requirements are:

If all of your input strings have a restricted vocabulary (it does not allow a free, random combination of letters, symbols and numbers, but it is restricted to a certain set of business words, identifiers and values), and also a simple grammar, you can try to design your own compress algorithm. Example:

input symbol compressed symbol ------------ ----------------- client 1 bill 2 date 3 amount 4 value 5 price 6 tax 7

If the grammar is simple but the vocabulary is not that restricted, you could perform an initial custom compression to compress the document's structure as much as you can, and then a second GZIP compression to compress the data.

And don't forget that you'll have to bundle the client application with the corresponding uncompressor.

Anyway, it's not an easy task, I admit it.

Deliver the response to the client application in streaming. If the protocol is HTTP, you could use Chunked Transfer Coding.
If everything else fails, you'll have to page the results and serve them to the client by pages on demand: The client makes a query, the server executes it and delivers just the first page of results. Then, the client may chose to read the next page.

Java - Compress large string

1 Answers1