Filter long log parameters in Rails

Question

I allow users to upload files on my site. And some of those files can be very large and it eats up a huge chunk of my log files. So I would like to not have it show up. I know about:

config.filter_parameters += [:password]

To filter certain parameters. But the problem with this is that it the parameter is in a hash like this:

{
   :person => { 
      :name => 'bob', 
      :file => { 
         :data => 'really long data. this can be tens of thousands of characters long' 
      }
   }
}

I could add :data to the filter_parameters but that would hiding lots of logs across the whole site since data is a common key (I also can't rename this to something more obscure). Is it possible for filter_parameters to take in a nested parameter? Or is there another way to limit the length of all parameters so if they come in bigger than a certain size, it would not be stored in my log files.

Why would uploading a large file use up a lot of log? It should be a single entry saying it was uploaded. — the Tin Man, Oct 21 '14 at 20:38
I guess I should've worded this better. It's not coming from my own website. Another website is posting it through a route on my site. It comes along with lots of other attributes and a file is one of the many things it's sending me. — Dragonfly, Oct 21 '14 at 20:43
It would probably help a lot if you showed a minimal example of the file so we can see it, not try to imagine it. — the Tin Man, Oct 21 '14 at 20:48
I updated it a bit if it helps. But basically there is this really long params that can be tens of thousands of lines long because of a large file. And it's killing my logs — Dragonfly, Oct 21 '14 at 22:33
Did you figure out a way to do this? trying to find the same. In my case, the `:data` is an image as base64 text. — mehulkar, Jan 16 '15 at 19:51
I couldn't find a really good way to do this but I I hacked around it. I posted the answer below. Hopefully it can help you or maybe you can find a better way to do this. — Dragonfly, Jan 16 '15 at 20:53
I think this is fairly normal thing to want to do. I work daily with REST APIs that accept base64 strings for files and the logs get impossible to read. — ichigolas, Feb 17 '16 at 15:16

score 10 · Accepted Answer · answered Jan 16 '15 at 20:52

I ended up putting something like this in my application.rb

config.filter_parameters << lambda do |k, v|
  if k == 'data' && v && v.class == String && v.length > 1024
    v.replace('[FILTER]')
  end
end

I couldn't find a better way to do this. So I look for the key 'data' in the params. And if the value for that data is a String and over a certain length, I just replace it so the logs isn't so cluttered.

score 2 · Answer 2 · edited Jun 20 '20 at 09:12

Since Rails 5.0, filter_parameters supports filtering for nested parameters using a dot notation.

In this case it would be:

config.filter_parameters += ['person.file.data']

There is an example in the documentation and at the actionpack CHANGELOG:

Add ability to filter parameters based on parent keys.
# matches {credit_card: {code: "xxxx"}}
# doesn't match {file: { code: "xxxx"}}
config.filter_parameters += [ "credit_card.code" ]
See #13897.

Filter long log parameters in Rails

2 Answers2

Linked