33

I want to store JSON data in local storage. Sometimes data stored could be more than 5MB(max threshold allowed by browsers for each domain). Is there anyway i can compress or zip the data and store it in local storage ? How much latency it will add if compression and decompression for every JS function on large data ?

I am using this json data to display on webpage..... It is like a static data which is mostly same for entire session. we provide actions like search, filter etc on this json data...Whenever user enters a keyword, i search in this json data which is stored in local storage instead of making call to server

Rajeev
  • 4,762
  • 8
  • 41
  • 63
  • What do you need this JSON data stored for? – Qantas 94 Heavy Dec 25 '13 at 15:04
  • I am using this json data to display on webpage..... It is like a static data which is same for entire session. we provide actions like search, filter etc on this json data...Whenever user enters a keyword, i search in this json data which is stored in localstorage instead of making call to server – Rajeev Dec 25 '13 at 15:39
  • I highly doubt it. Local storage was not designed to hold large amounts of data. Why not store this data on the server's cache? – anar khalilov Dec 25 '13 at 16:01
  • It seems rather exhaustive to unload everything you've got onto an unsuspecting user... 5MB? No amount of optimisation will compensate for those bandwidth implications - transfer just what you need when you need it in small chunks via ajax. – Emissary Dec 25 '13 at 16:03
  • @Emissary - It is very less possibility of crossing size more than 5MB. Most of the times, i am serving the same response. For this scenario, why to send another request to server when i can serve from local storage? First time i want to get the data through ajax calls only. – Rajeev Dec 25 '13 at 16:14

1 Answers1

30

As localStorage functionality seems to be limited to handle only string key/value pairs what you can do is to use lz-string library to stringify the json object and compress it before storing it in localStorage

Usage

// sample json object
var jsonobj = {'sample': 'This is supposed to be ling string', 'score': 'another long string which is going to be compressed'}

// compress string before storing in localStorage    
localStorage.setItem('mystring', LZString.compress(JSON.stringify(jsonobj)));

// decompress localStorage item stored
var string = LZString.decompress(localStorage.getItem('mystring'))

// parse it to JSON object
JSON.parse(string);

Check jsfiddle http://jsfiddle.net/raunakkathuria/7PQtC/1/ as i have copied the complete script so look for running code at end of fiddle

Use compressToUTF16 and decompressFromUTF16 if you are dealing with UTF characters but please note that the strings produced are therefore 6.66% bigger than those produced by compress. So use utf ones only when required.

Raunak Kathuria
  • 3,185
  • 1
  • 18
  • 27
  • Do u suggest any ecryption algorithm for morphing before storing in local storage? – Rajeev Dec 25 '13 at 16:15
  • There is one http://code.google.com/p/crypto-js/#CryptoJS_3.1 but have not used any so cant suggest – Raunak Kathuria Dec 25 '13 at 23:09
  • 8
    It says on the lz-string website that `compress` produces invalid UTF16 strings and will only work on webkit browsers. `compressToUTF16` and `compressFromUTF16` should be used for cross-browser compatibility. – mishmash Sep 04 '16 at 18:26
  • Thanks @vanneto, updated the answer to reflect your comment. – Raunak Kathuria Jan 31 '20 at 04:08