Can't filter sensitive data with VCR

Question

I've got the following in my spec_helper

c.filter_sensitive_data("<FILTERED>") { keys['s3_key'] }
c.filter_sensitive_data("<REDACTED>") { keys['s3_secret'] }

Yet when I run my spec I find that it creates the following entry in the cassette:

Authorization:
- AWS <FILTERED>:this_part_has_not_been_filtered=

As you can see there is a part that has not been filtered. I'm not sure if it contains useful information, but I don't want to paste it just incase. I can however say that it doesn't contain my key or my secret. Is it just fluff? Should I care? Is this what normally happens when filtering S3 requests when using the aws-sdk gem? If not then how can I get it to filter all of the authorization data?

Is there a special set of instructions for filtering S3 keys? I really don't want to mess this up.

score 2 · Accepted Answer · answered Aug 05 '15 at 20:46

Well, looks like it's safe, if your key isn't there. To be sure, you might use regexp matcher to replace whole string, something like %r<#{keys['s3_key']:.*?=>. Bad news: there are no regexp filter_sensitive_data. Good news: you can use more low-level methods to implement that yourself.

That's current implementation of filter_sensitive_data

# @param placeholder [String] The placeholder string.
# @param tag [Symbol] Set this to apply this only to cassettes
#  with a matching tag; otherwise it will apply to every cassette.
# @yield block that determines what string to replace
# @yieldparam interaction [(optional) VCR::HTTPInteraction::HookAware] the HTTP interaction
# @yieldreturn the string to replace
def define_cassette_placeholder(placeholder, tag = nil, &block)
  before_record(tag) do |interaction|
    orig_text = call_block(block, interaction)
    log "before_record: replacing #{orig_text.inspect} with #{placeholder.inspect}"
    interaction.filter!(orig_text, placeholder)
  end

  before_playback(tag) do |interaction|
    orig_text = call_block(block, interaction)
    log "before_playback: replacing #{placeholder.inspect} with #{orig_text.inspect}"
    interaction.filter!(placeholder, orig_text)
  end
end
alias filter_sensitive_data define_cassette_placeholder

Source

Which leads us to these methods

  # Replaces a string in any part of the HTTP interaction (headers, request body,
  # response body, etc) with the given replacement text.
  #
  # @param [#to_s] text the text to replace
  # @param [#to_s] replacement_text the text to put in its place
  def filter!(text, replacement_text)
    text, replacement_text = text.to_s, replacement_text.to_s
    return self if [text, replacement_text].any? { |t| t.empty? }
    filter_object!(self, text, replacement_text)
  end

private

  def filter_object!(object, text, replacement_text)
    if object.respond_to?(:gsub)
      object.gsub!(text, replacement_text) if object.include?(text)
    elsif Hash === object
      filter_hash!(object, text, replacement_text)
    elsif object.respond_to?(:each)
      # This handles nested arrays and structs
      object.each { |o| filter_object!(o, text, replacement_text) }
    end

    object
  end

Source

Oh well, we might just try monkey patching this method:

Somewhere in your spec_helper:

class VCR::HTTPInteraction::HookAware
  def filter!(text, replacement_text)
    replacement_text = replacement_text.to_s unless replacement_text.is_a?(Regexp)
    text = text.to_s
    return self if [text, replacement_text].any? { |t| t.empty? }
    filter_object!(self, text, replacement_text)
  end
end

Of course, you can just opt out messing with the deep internals of alien library, and don't feel too paranoid knowing that some random alpha-numeric data is written to cassette near your token (but not including the latter).

Can't filter sensitive data with VCR

1 Answers1