29

I have an api written in rails which on each request responds with a JSON response.

The response could be huge, so i need to compress the JSON response using gzip.

Wondering how to do this in rails controller?

I have added the line

use Rack::Deflater

in config.ru

Should I also be changing something in the line which renders JSON?

render :json => response.to_json()

Also, how do i check if the response is in gzip format or not..??

I did a curl request from terminal, I see only the normal plain JSON.

aBadAssCowboy
  • 2,440
  • 2
  • 23
  • 38

5 Answers5

37

My post Content Compression with Rack::Deflater describes a couple of ways to integrate Rack::Deflater. The easiest would be to just update config/application.rb with:

module YourApp
  class Application < Rails::Application
    config.middleware.use Rack::Deflater
  end
end

and you'll automatically compress all controller responses with deflate / gzip if the client explicitly says they can handle it.

djcp
  • 699
  • 1
  • 6
  • 6
  • I needed this Rack Middleware declaration as well as the headers to get a gzip response. Thanks! – Nate Bird Nov 18 '13 at 16:38
  • I add `config.middleware.use Rack::Deflater` to application.rb, compression start working only on Mac OS (Chrome, FF and Safary). On windows I receive responses without 'Content-Encoding: gzip;' in FF and IE (works only in chrome). P.S. "Accept-Encoding: gzip" present in all browsers. Do you know why?) – bmalets May 19 '15 at 11:22
  • @bmalets Does this still happen? Found a way around this? – Christian Fazzini Dec 23 '15 at 15:42
  • @ChristianFazzini, as I remember, it was a problem of old FF and IE browsers and I fixed this problem by enabling gzipping in nginx configurations – bmalets Dec 23 '15 at 15:47
  • @bmalets Do you remember the versions of FF and IE you are referring to? About adding gzipping in nginx configurations. Is it something along the lines of https://mattstauffer.co/blog/enabling-gzip-on-nginx-servers-including-laravel-forge – Christian Fazzini Dec 23 '15 at 16:00
  • @ChristianFazzini please, take a look at http://stackoverflow.com/questions/30323996/http-compression-in-rails-not-working-for-json-responses – bmalets Dec 23 '15 at 16:05
14

For the response to be in gzip format we don't have to change the render method call.
If the request has the header Accept-Encoding: gzip, Rails will automatically compress the JSON response using gzip.

If you don't want the user to send a request with preset header., you can add the header to the request manually in the controller before rendering the response:

request.env['HTTP_ACCEPT_ENCODING'] = 'gzip'
render :json => response.to_json()
aBadAssCowboy
  • 2,440
  • 2
  • 23
  • 38
  • Confirming @curiousmind's response below - you also have to add the `Rack::Deflater` middleware to get Rails to compress the JSON response, _even if_ you have passed the Accept-Encoding request header in set to 'gzip,deflate'. Note that the request header is also required for compression to kick in. – sameers Mar 19 '15 at 17:32
5

You can query Curl by setting a custom header to get gzipped response

$ curl -H "Accept-Encoding: gzip, deflate" localhost:3000/posts.json > posts_json.gz

then, then decompress it to view the actual response json

 $ gzip -d posts_json.gz
 $ cat posts_json

If it doesn't work. post back with output of rake middlewares to help us troubleshoot further.

CuriousMind
  • 33,537
  • 28
  • 98
  • 137
  • 3
    Along with the headers I also had to add 'config.middleware.use Rack::Deflater' to the app's application.rb file for it to actually compress the file. – Nate Bird Nov 18 '13 at 16:37
1

In some cases you can consider to write huge response into a file and gzip it:

res = {} # huge data hash
json = res.to_json

Zlib::GzipWriter.open('public/api/huge_data.json.gz') { |gz| gz.write json }

and update this file regularly

Lev Lukomsky
  • 6,346
  • 4
  • 34
  • 24
1

Consider not putting Rack middlewares in config.ru when using Rails

Rails has it's own middleware stack manager since Rails 2.

The correct way is:

# config/application.rb or config/environment.rb depends on your Rails version
config.middleware.use Rack::Deflater

Don't use @djcp's solution when using Rack::ETag

Short answer:

module MyApp
  class Application < Rails::Application
    config.middleware.insert_before Rack::ETag, Rack::Deflater
  end
end

The order of Rack::Deflater and Rack::ETag matters because Rack::Deflater uses Zlib::GzipWriter to compress the response body and it would compress with a timestamp by default, which means the compressed response body would change every second even if the original response body is the same.

To reproduce this problem, run the following script:

require 'rack/etag'
require 'rack/deflater'
require 'rack/content_length'

@app = Rack::Builder.new do
  use Rack::ETag
  use Rack::Deflater
  use Rack::ContentLength
  run ->(*) { [200, {}, ['hello world']] }
end

def puts_etag
  puts @app.call({ 'HTTP_ACCEPT_ENCODING' => 'gzip' })[1]['ETag']
end

puts_etag
sleep 1
puts_etag

One can simply swap the lines of Rack::ETag and Rack::Deflater and get the expected output.

Rails uses Rack::ETag by default and config.middleware.use is just appending. To insert Rack::Deflater before Rack::Etag, use config.middleware.insert_before instead.

Weihang Jian
  • 7,826
  • 4
  • 44
  • 55