FastAPI and Celery

Learning distributed task queues by doing. Since it's a greenfield project this also uses a newer async web framework.

See: Celery docs

Debugging Celery

Check results in redis result backend

docker-compose exec redis sh
redis-cli
KEYS celery*
MGET celery-task-meta-<UID of Task>

Checking results in Flower

Use the flower dashboard at: 0.0.0.0:5557

Eager Task Processing

CELERY_TASK_ALWAYS_EAGER: bool = True will synchronously execute the celery tasks. This allows us to use things like breakpoint() to enter a debugger within the execution context of the task.

Since the app currently uses the fastapi application config for celery config, add CELERY_TASK_ALWAYS_EAGER=True to the config class if needed. You don't need to run the worker, message broker, or result backend processes to debug your code. It will process the task within