How to combine multiple QuerySets in Django?

ghz 1years ago ⋅ 8364 views

Question

I'm trying to build the search for a Django site I am building, and in that search, I am searching across three different models. And to get pagination on the search result list, I would like to use a generic object_list view to display the results. But to do that, I have to merge three QuerySets into one.

How can I do that? I've tried this:

result_list = []
page_list = Page.objects.filter(
    Q(title__icontains=cleaned_search_term) |
    Q(body__icontains=cleaned_search_term))
article_list = Article.objects.filter(
    Q(title__icontains=cleaned_search_term) |
    Q(body__icontains=cleaned_search_term) |
    Q(tags__icontains=cleaned_search_term))
post_list = Post.objects.filter(
    Q(title__icontains=cleaned_search_term) |
    Q(body__icontains=cleaned_search_term) |
    Q(tags__icontains=cleaned_search_term))

for x in page_list:
    result_list.append(x)
for x in article_list:
    result_list.append(x)
for x in post_list:
    result_list.append(x)

return object_list(
    request,
    queryset=result_list,
    template_object_name='result',
    paginate_by=10,
    extra_context={
        'search_term': search_term},
    template_name="search/result_list.html")

But this doesn't work. I get an error when I try to use that list in the generic view. The list is missing the clone attribute.

How can I merge the three lists, page_list, article_list and post_list?

Answer

Concatenating the querysets into a list is the simplest approach. If the database will be hit for all querysets anyway (e.g. because the result needs to be sorted), this won't add further cost.

from itertools import chain
result_list = list(chain(page_list, article_list, post_list))

Using itertools.chain is faster than looping each list and appending elements one by one, since itertools is implemented in C. It also consumes less memory than converting each queryset into a list before concatenating.

Now it's possible to sort the resulting list e.g. by date (as requested in hasen j's comment to another answer). The sorted() function conveniently accepts a generator and returns a list:

from operator import attrgetter
result_list = sorted(
    chain(page_list, article_list, post_list),
    key=attrgetter('date_created')
)

You can reverse the sort order:

result_list = sorted(
    chain(page_list, article_list, post_list),
    key=attrgetter('date_created'),
    reverse=True,
)

attrgetter is equivalet to the following lambda (this was the way it had to be done before Python 2.4):

result_list = sorted(
    chain(page_list, article_list, post_list),
    key=lambda instance: instance.date_created,
)
#django