Skip to content

Worker child is not restarted after --max-tasks-per-child is reached #646

Description

@aysanorhun

Taskiq version

0.12.4

Python version

Python 3.12

OS

Linux

What happened?

Description

With Taskiq 0.12.4 and taskiq-redis 1.2.3, when --max-tasks-per-child is reached, the worker stops prefetching, waits for running tasks, and starts broker shutdown:

Max number of tasks executed.
Stopping prefetching messages...
No more tasks to wait for. Shutting down.
Shutting down the broker.

However, the child process remains alive. Since ProcessManager only restarts a worker after detecting that it is dead, this worker slot is never respawned and stops consuming tasks.

The only workaround I found was forcing the child to exit using os._exit(...), after which ProcessManager correctly respawned that specific worker.

Expected behavior

The child should exit after completing its admitted tasks, allowing ProcessManager to respawn it.

The documentation also describes this as a restart:

--max-tasks-per-child - maximum number of tasks to be executed by a single worker process before restart.

https://taskiq-python.github.io/guide/cli.html#other-parameters

The option was originally requested in #309 and implemented by #314. Issue #556 also shows the expected worker is dead → scheduling reload → restarted sequence.

Is the intended behavior for the child to exit and be respawned after reaching --max-tasks-per-child? If so, should the worker guarantee termination when graceful receiver or broker shutdown does not complete?

I previously created a discussion about this behavior:

https://github.com/orgs/taskiq-python/discussions/608

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions