Celery is a distributed task queue written in Python.

learn more… | top users | synonyms

1
vote
1answer
42 views

Is there a way to use length of a RabbitMQ queue used by Celery to start instance in an autoscale group?

Is there any way for Celery to emit events when the length of a queue exceeds a threshold? I want to use that event to start an EC2-instance. We have two queues for two different tasks in Celery. ...
1
vote
0answers
24 views

Supervisor start/stop/restart not behaving as expected

I am using supervisor to manage celery and due to occasional project requirements am required to add environment variables. I set these in the supervisor [environment] section of the .conf file but ...
0
votes
0answers
34 views

Celery SQS workers slow

I am testing Celery and its SQS implementation. I do realise that its under development. My findings are that i am able to push hundreds of messages a second, however my workers are unable to consume ...
0
votes
1answer
94 views

Is there a more convenient way to stop (or restart) a detached celery beat process?

Just to clarify things - I am working on a systemd service file celerybeat.service which should take care of the Celery beat. What I am doing at the moment is calling a script which reads ...
0
votes
0answers
74 views

Celery worker timeouts

I'm trying to run a celery worker with each task having a timeout of 30 seconds. However whenever I run celery with the --time-limit option, it simply quits. This is the command I use. python ...
0
votes
0answers
277 views

Run shell script as Supervisord subprocess?

I have a Django 1.6.2 application that uses Celery 3.1.7 for asynchronous tasks. I'm starting my Celery workers using Supervisor. So far everything is working well except when I reboot my Debian 7.8 ...
0
votes
0answers
37 views

More than one celery server sharing the same message queue?

I have a number of servers all running the same celery app to process tasks. I am using SQS as the message queue. Right now, each app appears to be automatically creating its own queue when it ...
0
votes
0answers
43 views

VPN-tunnel renegotiation problems

I have an application running on top of Celery with redis as a broker. Redis is hosted at our datacenter but the workers are hosted on a separate line. We have setup two-way communcation between the ...
0
votes
1answer
165 views

Running multiple workers using Celery

I need to read from Rabbitmq and execute task in parallel using Celery in a single system. [2014-12-30 15:54:22,374: INFO/Worker-1] ... [2014-12-30 15:54:23,401: INFO/Worker-1] ... [2014-12-30 ...
1
vote
1answer
350 views

Supervisord as Windows Service on Cygwin

I am attempting to run Celery as a Windows Service using Supervisord. I followed the configuration laid out on the Celery site and here. I have set up a virtual environment to run supervisord through ...
1
vote
2answers
1k views

Celery with Upstart - processes are dying unexpectedly

When I'm running Celery with Upstart, after a while, the child processes or the main processes die without any trace. The Upstart script I'm using (/etc/init/celery): description "celery" start on ...
0
votes
1answer
180 views

celery tasks from one domain appear on another domain's django admin hosted on the same box

I have a rather funny problem that I didn't manage to find it elsewhere. I have 2 django-celery projects running with redis on 2 different domains, but apparently hosted on the same box. I don't have ...
0
votes
1answer
850 views

Celery and RabbitMQ - Tools for monitoring and callout alarms

I think we've reached the limit of usability of celery events and celery fllower. Is there a good tool for monitoring status of Celery running under RabbitMQ? In particular, I would like something ...
0
votes
1answer
875 views

Approach to auto scale celery servers based on broker(redis) queue size

I am working on a project which requires rolling out new celery servers if the broker(redis) queue is consistently higher than a predetermined threshold size and killing the new boxes when the queue ...
1
vote
2answers
1k views

kill -HUP is not working with celery daemon

So I have a shell script that daemonizes celery, and creates a bunch of workers running as a daemon. I would like to have a way to restart the celery task when the underlying source gets changed since ...
7
votes
1answer
2k views

Celery Daemon receives unregistered tasks

I installed Celery for my Django project following what the official tutorial / doc says. And it's working fine when I launch celery at the command line, I can see it receiving the tasks and execute ...
1
vote
0answers
606 views

OSError : [Errno 38] Function not implemented - Django Celery implementation

I installed django-celery and I tried to start up the worker server but I get an OSError that a function isn't implemented. I'm running CentOS release 5.4 (Final) on a VPS: . broker -> ...