Django and Celery
Celery is a python-based open-source library. It is used to do tasks asynchronously. It is mainly used in large-scale projects where servers had to handle heavy computation. In general terms, it is a task queue that holds the tasks and distributes them to workers. The main use of celery is to do real-time operations but we can also use this to schedule tasks. One of the main advantages of using celery is to send responses to our clients fastly so that the users don’t have to wait for a long time.
What are message brokers?
Celery uses a third-party message service to send and receive messages. These separate message services are termed message brokers. In celery, we generally use Redis or RabbitMQ to convey messages between the client and celery.
Advantages of using Celery -
-
Celery is an open-source library so we don’t have to pay anything to use it.
-
Celery is able to process millions of tasks in a single minute.
-
It is simple to use.
-
It supports multiple message brokers such as Redis, RabbitMQ, etc.
-
We can also use celery to schedule tasks.
In this tutorial, we’ll use celery to send emails to our users without any loading.
Note - We hope you have already done the set-up of your Django Project and app file. If not you can follow our previous tutorials to do that.
So, without wasting any time let's get started by installing celery.
Install Celery
Use the following command to install celery.
Pip install celery
Create Celery file
Create a celery.py file inside your project file and add the following code in your celery.py file.
ProjectFile/celery.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'ProjectFileName.settings')
app = Celery('ProjectFileName')
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')
# Load task modules from all registered Django apps.
app.autodiscover_tasks()
@app.task(bind=True)
def debug_task(self):
print(f'Request: {self.request!r}')
You have to replace the ProjectFileName by your project name. If you want to understand the above code line by line read the official documentation of celery.
Add Configurations in _init_.py file
ProjectFile/_init_.py
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app
__all__ = ('celery_app',)
Here, we have imported our app from celery.py file as celery_app.
Install Redis
In this tutorial we’ll use redis as a message broker. Use the following command to install redis.
pip install redis
After installing redis we have to download redis to use its CLI. you can visit the github link https://github.com/tporadowski/redis/releases to download msi.
Click on Redis-x64-5.0.14.1.msi and install it.
After redis gets installed successfully open Program Files inside your c drive folder and open redis CLI and enter the ping command to check it is properly working or not.
If you get pong in response then your redis is working properly.
Update Settings
To use celery we have to add some configurations to our settings.py file. Add the configurations given below.
ProjectFile/settings.py
CELERY_BROKER_URL = 'redis://127.0.0.1:6379'
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_SERIALIZER = 'json'
Create task.py file
Now, create a task.py file in your app file. As we are creating a project to send emails therefore, we are creating a task appropriate to our project. You can create task according to your use.
AppFile/task.py
from celery import shared_task
from django.core.mail import send_mail
@shared_task
def send_mail_task():
send_mail(
'Testing Django Celery',
'this main is sended by codersdaily admin with Django celery',
'totalgamingg01@gmail.com',
['rohit.sharma@techsolvo.com'],
fail_silently=False,
)
return None
Here, i have imported shared_task from celery and send_mail to use Django’s mail facility. Shared_task is a decorator that allow the creation of celery task for reusable apps. Then we created a function send_mail_task. Inside send_mail we have added the title, message, sender’s mail address, and receiver’s mail address. As we want to send emails we have to add some settings.
Add Mail Settings
ProjecFile/settings.py
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_USE_TLS = True
EMAIL_PORT = 587
EMAIL_HOST_USER = 'YourUserID'
EMAIL_HOST_PASSWORD = 'Password*****'
EMAIL_USER_SSL = False
Here, we’ve added all required settings. Sometimes we get problem in using celery on windows. To solve this we’ll install a requirement.
Install gevent
Use the following command to install gevent.
Pip install gevent
Create views.py file
AppFile/views.py
from.task import *
def mail(request):
send_mail_task.delay()
return HttpResponse("<h1> Sending without celery </h1>")
Here, we have imported everything from our task file. Then we created a function mail and added our task. Delay is a shortcut in celery to send a task message.
Run celery server
Now create two terminals. In first terminal run Django server and in second server run celery server by using the given command.
celery -A YourCeleryAppName worker -l info -P gevent
After running this command you will get a interface like this.
So, we have done almost all the setup now let's check whether our project is working or not and how much time it will take to send emails.
Our webpage is refreshed in less than a second. Now let's check if we got an email or not.
Successfully Got!
