How do you split a Django queryset without evaluating it?

Django provides a few classes that help you manage paginated data – that is, data that’s split across several pages, with “Previous/Next” links:

from django.core.paginator import Paginator

object_list = MyModel.objects.all()
paginator = Paginator(object_list, 10) # Show 10 objects per page, you can choose any other value

for i in paginator.page_range(): # A 1-based range iterator of page numbers, e.g. yielding [1, 2, 3, 4].
    data = iter(paginator.get_page(i))
    # use data

If your django version is 1.11 or less than that like 1.10, 1.9 or so on, then use paginator.page(page_no) but be careful that this may raise an InvalidPage Exception when invalid/no page has been found.

For versions <= 1.11, use below code:

from django.core.paginator import Paginator

qs = MyModel.objects.all()
paginator = Paginator(qs, 20)

for page_no in paginator.page_range:
    current_page = paginator.page(page_no)
    current_qs = current_page.object_list

And if you're using django version >= 2.0, please use paginator.get_page(page_no) instead, but you can also use paginator.page(page_no).

For versions >= 2.0, use below code:

from django.core.paginator import Paginator

qs = MyModel.objects.all()
paginator = Paginator(qs, 20)

for page_no in paginator.page_range:
    current_page = paginator.get_page(page_no)
    current_qs = current_page.object_list

The advantage of using paginator.get_page(page_no) according to django documentations is as follows:

Return a valid page, even if the page argument isn't a number or isn't in range.

While in the case of paginator.page(page_no), you have to handle the exception manually if page_no is not a number or is out of range.

Passing query sets to Threads is not something I would recommend. I know the sort of thing you are trying to do and why, but its best to just pass some sort of param set to each thread and then have the Thread perform the partial query. Working this way, your threads are distinct from the calling code.

On a different note, if you are trying to use threads as a work around for the lags caused by high DB queries, you might find using transaction management a better route. This link link has some useful tips. I use this instead of Threads