Extend custom Rails preloader with includes

Question

I'd like to trigger eager loading on a relation with an #includes call, that was modified with a custom preloader using ActiveRecord::Associations::Preloader on a Rails 7 app, Ruby 3.2:

class Product < ApplicationRecord
  belongs_to :vendor
  has_many :taxons
end

class Vendor < ApplicationRecord
  has_many :products
end

records = Product.where(id: [1,2,3,4])
scope = Vendor.where.not(id: [5,6])
preloader = ActiveRecord::Associations::Preloader.new(records:, associations: :vendor, scope:).tap(&:call)

puts records.first.association_cached?(:vendor)
=> true

puts records.includes(:taxons).first.association_cached?(:vendor)
=> false

I need the scope: argument because some users aren't allowed to access certain resources.

Without the includes(:taxons) call, the association :vendor is correctly eager loaded and cached. But in the second case, it seems like the whole eager loading of Preloader is somehow dropped.

Is there a way to include a custom preloader as a chain argument along with other includes? I get an error when I try to include the preloader itself like this:

records.includes(preloader).to_a
ActiveRecord::AssociationNotFoundError: Association named '#<ActiveRecord::Associations::Preloader:0x00007f573bc31ab0>' was not found on Product; perhaps you misspelled it?
from /usr/local/bundle/gems/activerecord-7.0.4.2/lib/active_record/associations.rb:302:in `association'

score 2 · Answer 1 · answered Feb 15 '23 at 19:20

What's happening here is that ActiveRecord::Associations::Preloader needs the objects to be initialized to fill in the data however calling any queries on ActiveRecord::Relation will reload the objects.

You have to call the Preloader after calling any reload causing method of ActiveRecord::Relation on records. It shouldn't have any performance difference from adding the custom preloader to the chain since it still would have to be called each time at the end.

Adding it to the chain would require some monkey patching. The starting point for it would be ActiveRecord:: Relation#preload_associations to where you have to pass the scope.

If you just need the data, call includes first, then Preloader. I you need the data first without taxons and later include them, call a second custom preloader to load the taxons data. It will also have a positive impact on speed since products and vendors don't have to be loaded and initialized again.

records = Product.where(id: [1,2,3,4])
scope = Vendor.where.not(id: [5,6])
ActiveRecord::Associations::Preloader.new(records:, associations: :vendor, scope:).tap(&:call)

# associations filtered by the scope are initialized but return nil, although association_id is present
records.map{|r| r.association_cached?(:vendor)}
=> [true, true, true, true]
records.map{|r| r.association_cached?(:taxons)}
=> [false, false, false, false]

ActiveRecord::Associations::Preloader.new(records:, associations: :taxons).tap(&:call)

records.map{|r| r.association_cached?(:vendor)}
=> [true, true, true, true]
records.map{|r| r.association_cached?(:taxons)}
=> [true, true, true, true]

Unfortunately, the `ActiveRecord::Associations::Preloader` is always called before the `include`, due to the structure of the code. So I guess I have to settle with a monkey patch that has not too much impact. Thanks for your answer and pointing me in the right direction where to put the monkey patch :) — 23tux, Feb 16 '23 at 06:22

score 2 · Accepted Answer · answered Feb 16 '23 at 06:29

As Jan Vítek pointed out, any queries called on a relation will reload all objects, and therefore loose all already eager loaded associations.

Due to the structure of the code, I can't guarantee to always call the custom ActiveRecord::Associations::Preloader, so I came up with a little monkey patch for ActiveRecord::Relation#preload_associations:

raise "ActiveRecord::Relation#preload_associations is no longer available, check patch!" unless ActiveRecord::Relation.method_defined? :preload_associations
raise "ActiveRecord::Relation#preload_associations arity != 1, check patch!" unless ActiveRecord::Relation.instance_method(:preload_associations).arity == 1

module PreloadWithScopePatch
  def preload_with_scope(association, scope)
    scope = model.reflect_on_association(association).klass.where(scope) if scope.is_a?(Hash)
    (@preload_with_scope ||= []).push([association, scope])
    self
  end

  def preload_associations(records)
    super.tap do
      Array(@preload_with_scope).each do |associations, scope|
        ActiveRecord::Associations::Preloader.new(records:, associations:, scope:).call
      end
    end
  end
end
ActiveRecord::Relation.prepend(PreloadWithScopePatch)
ActiveRecord::Base.singleton_class.delegate(:preload_with_scope, to: :all)

Now you can use it like this:

records = Product.where(id: [1,2,3,4]).preload_with_scope(:vendor, Vendor.where.not(id: [5,6])).includes(:taxons)

records.map { |r| r.association_cached?(:vendor) }
=> [true, true, true, true]

records.map { |r| r.association_cached?(:taxons) }
=> [true, true, true, true]

Extend custom Rails preloader with includes

2 Answers2