0

I'm running a Rails app (v 3.1.10) on a Heroku Cedar stack with Papertrail add-on going crazy because of the size of the logs.

My app is really verbose and the logs are getting huge (really huge):

Sometimes because I serialize a lots of data in one field and that makes a huge SQL request. In my model I have many:

serialize :a_game_data, Hash
serialize :another_game_data, Hash
serialize :a_big_set_of_game_data, Hash
[...]

Thanks to my AS3 Flash app working with bigs sets of json...

Sometimes because there's a lots of partials to render:

Rendered shared/_flash_message.html.erb (0.1ms)
Rendered shared/_header_cart_info.html.erb (2.7ms)
Rendered layouts/_header.html.erb (19.4ms)
[...]

It's not the big issue here, but I've added this case too because Jamiew handle it, see below...

Sometimes because there's lots of sql queries on the same page:

User Load (2.2ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
Course Load (5.3ms)  SELECT "courses".* FROM "courses" WHERE (id = '1' OR pass_token = NULL)
Session Load (1.3ms)  SELECT "sessions".* FROM "sessions" WHERE "sessions"."id" = 1 LIMIT 1
Training Load (1.3ms)  SELECT "trainings".* FROM "trainings" WHERE "trainings"."id" = 1 LIMIT 1
[...]

It's a big (too) complex App we've got here... yeah...

Sometimes because there's a lots of params:

Parameters: {"_myapp_session"=>"BkkiJTBhYWI1MUVlaVdtbE9Eb1Y2I5BjsAVEkiEF9jc3JmX3Rva2VlYVWZyM2I0dEZaR1YwNXFjZhZTQ1uBjsARkkiUkiD3Nlc3Npb25faWQGOgZFRhcmRlbi51c2yN1poVm8vdWo3YTlrdUZzVTA9BjsARkkiH3dAh7CMTQ0Yzc4ZDJmYzg5ZjZjOGQ5NVyLmFkbWluX3VzZXIua2V5BjsAVFsISSIOQWRtaW5Vc2VyBjsARlsGaQZJIiIkMmEkMTAkcmgvQ2Rwc0lrYzFEbGJFRG9jMnZvdQY7AFRJIhl3YXJkZW4udXNlci51c2VyLmtleQY7AFRbCEkiCVVzZXIGOwBGWwZpBkkiIiQyYSQxMCRBUFBST2w0aWYxQmhHUVd0b0V5TjFPBjsAVA==--e4b53a73f6b622cfe7550b2ee12678712e2973c7", "authenticity_token"=>"EeiWmlODoYXUfr3b4tFZGV05qr7ZhVo/uj7a9kuFsU0=", "utf8"=>"✓", "locale"=>"fr", "id"=>"1", "a"=>1, "a"=>1, "a"=>1, "a"=>1, "a"=>1, "a"=>1, [...] Hey! You've reach the end of the line but it's not the end of the parameters...}

The AS3 Flash app send big json data to the controller...

I didn't mention the (in)famous "Assets pipeline logging problem" because now I'm using the quiet_assets gem to handle this:
https://github.com/evrone/quiet_assets

So... what did I try?

1: Dennis Reimann's middleware solution:
http://dennisreimann.de/blog/silencing-the-rails-log-on-a-per-action-basis/

2: Spagalocco's gem (inspired by solution #1):
https://github.com/spagalloco/silencer

3: jamiew's monkeypatches (inspired by solution #1 + a bonus):
https://gist.github.com/1558325

Nothing is really working as expected but it's getting close.

I would rather use a method in my ApplicationController like this: 

def custom_logging(opts={}, show_logs=true)
  disable_logging unless show_logs
  remove_sql_requests_from_logs if opts[:remove_sql_requests]
  remove_rendered_from_logs if opts[:remove_rendered]
  remove_params_from_logs if opts[:remove_params]
  [...]
end

...and call it in any controller method: custom_logging({:remove_sql_requests=>1, :remove_rendered=>1})

You got the idea.

So, is there any good resource online to handle this? Many thanks for your advices...

fro_oo
  • 1,610
  • 4
  • 24
  • 46
  • did you find a solution that works to silence the serialization logging? That alone makes my production logs unusable. – jpw Dec 16 '13 at 08:02
  • Not really... I'm using 'quiet_assets' and 'lograge'. – fro_oo Mar 03 '14 at 17:27

2 Answers2

1

I"m the author of the silencer gem mentioned above. Are you looking to filter logging in general or for a particular action? The silencer gem handles the latter problem. While you can certainly use it in different ways, it's mostly intended for particular actions.

It sounds like what you are looking for less verbose logging. I would recommend you take a look at lograge. I use that in production in most of my Rails apps and have found it to be quite useful.

If you need something more specialized, you may want to look at implementing your own LogSubscriber which is essentially the lograge solution.

stve
  • 219
  • 2
  • 4
  • Indeed, I want to filter by actions and also specific info in the logs. Cool answer, thank you anno. I'll look at your gems and follow the links mentioned above... – fro_oo Jan 29 '13 at 10:39
0

Set your log level in the Heroku enviroment

View your current log level:

heroku config

You most likely have "Info", which is just a lot of noise

Change it to warn or error

heroku config:add LOG_LEVEL=WARN

Also, when viewing the logs, only specify the "app" server

heroku logs --source app

I personally, append --tail to see the logs live.

heroku logs --source app --tail
Andrew Wei
  • 2,020
  • 1
  • 17
  • 28
  • Thank you but it's not what I'm looking for. I've already set the log level. I don't need to filter the logs while reading them, but instead while writing them. Because as I've said in my post, Papertrail is going crazy :-) – fro_oo Jan 28 '13 at 20:44