I spent a frustrating hour debugging an issue: I was following Heroku’s excellent docs on deploying a Django application and got to the part about setting up a background worker using Celery and Kombu.I had two terminal tabs open for testing:
1) Tab 1 had celeryd and runserver.
2) Tab 2 had an ipython shell open
I added a task to the Celery queue using the Ipython shell. And then I waited. And waited. The task showed up in the MySQL table, but it was just not getting picked up by Celery. I terminated and restarted Celery, and lo and behold, the task would get processed. WTF.
An hour of experimentation and Googling gave me the result: “Django-celery’s database backend basically doesn’t work with innodb unless you change this [transaction isolation level] setting because django runs everything in transactions by default”.
Ouch.
In development, the solution is to close your ipython shell.
In production, change this setting.