I’m working on a Django application where I have a Question model containing around 500,000+ records.

I added a custom Django Admin action to run AI moderation on selected questions:

@admin.action(description="Run AI Moderation")
def run_ai_moderation(self, request, queryset):
    for question in queryset:
        result = moderate_question(question.title, question.content)
        question.moderation_score = result["score"]
        question.status = result["status"]
        question.save()

The action works correctly when only 5–10 questions are selected. However, when an admin selects hundreds or thousands of questions, the request becomes extremely slow and eventually returns a 504 Gateway Timeout.

There are also some additional problems:

• The AI API has rate limits.
• Some AI requests take several seconds to respond.
• If the process fails halfway through, previously processed questions are updated but the remaining questions are not.
• Running the action directly inside the Django request blocks the web server.
• Admin users sometimes click the action multiple times, causing duplicate AI requests.
• I don't want the Django web request to remain open while the moderation process is running.

Questions

What would be the production-ready Django solution for this problem?

Specifically:

1. Should I use Celery, Django Q, or another background task system?
2. How should I process thousands of questions without blocking the admin request?
3. How can I handle AI API rate limits and retries?
4. How can I prevent the same question from being moderated multiple times simultaneously?
5. Should I use bulk_update() or individual save() calls?
6. How should I track task progress and failures?
7. How can I make the admin action return immediately while the background process continues?
8. What database transaction or locking strategy should be used?
9. How should failed AI requests be retried safely?
10. What would be the recommended architecture for this in a production Django application?