The ugly? There is no ugly. Be grateful for the open source software you use! Understand software before making a choice and blame yourself if you made the wrong one, not the authors!
Our paradoxical choice We needed more from our task queue We needed to understand 100% of how it works We didn’t need layers of abstraction for everything. Just one implementation that worked! - = +
Quick comparison Celery RQ Broker RabbitMQ, Redis, MongoDB, ... Redis Concurrency Master-slave processes supervisord + fork() Lines of code ~27k ~2k Scheduler Built-in 3rd-party Routing Advanced exchanges List of queues
SIMPLE 3 main classes: Job, Queue, Worker Queue.enqueue_call() :
Still not perfect! Creator @nvie also the only primary author Much less mature (0.3.8) than Celery (3.0.21) Can’t serialize task results to JSON (only pickle) Limited concurrency model (for now) Monitoring tools can be improved
Our migration Surprisingly easy We didn’t use much of the advanced Celery features Just changed task parent class + send_task() Joy of understanding 100% of your app’s code ;) Discovered some long-standing errors in our tasks Only real step back was the monitoring frontend