Run a rails query in batches
.in_batches
The issue with find_each
or find_in_batches
is that you have consumed query results.
The cleanest solution is using in_batches
, because it yield the actual query (without consuming it):
User.find_in_batches do |users|
users.select(:id) # error
end
User.in_batches do |users|
users.select(:id) # works as expected
users.pluck("complext pluck query") # works as expected
end
You need to use find_each
with the option batch_size.
A.where(:name => "John").find_each(batch_size: 1000) do |a|
# your code
end
An alternative to using find_each
is to use find_in_batches
.
There's a distinct difference - find_each
will give your block each item and will loop through your batch item by item. find_in_batches
will deliver your batch of items in an array to your block.
I've assumed your A
model is actually called Address
. You could do something like this:
Address.where(name: "John").find_in_batches(batch_size: 1000) do |addresses|
# Your code that you might want to run BEFORE processing each batch ...
addresses.each do |address|
# Your code that you want to run for each address
end
# Your code that you might want to run AFTER processing each batch ...
end
As you can see, this gives you a little more flexibility around how you handle the processing of your batches. However, if your needs are simple, just stick with find_each
.