Regression on multiprocessing’s resource tracker in 3.13

[facundobatista]

Bug report

Bug description:

The resource tracker cleanup has a regression in 3.12.10 (was working 3.12.9, @colesbury found; it's possibly related to this PR).

This is a simple script to expose the problem. It creates a portion of shared memory, then forks. The child writes in that shared memory. The parent waits for the child to finish, reads the shared memory, do the proper cleanup, and then shows the result.

import os
import struct
from multiprocessing import shared_memory


type_size = struct.calcsize('B')
chunk_size = type_size * 2
shm = shared_memory.SharedMemory(create=True, size=chunk_size)

pid = os.fork()
if pid == 0:
    # child
    values = [1, 2]
    offset = 0
    struct.pack_into("2B", shm.buf, offset, *values)
    exit(0)

# only the parent will reach here
os.wait()

# get produced value and cleanup the shared memory
result = struct.unpack("2B", shm.buf[:chunk_size])  # buffer size may be larger than indicated
shm.close()
shm.unlink()

print(result)

It works just fine in 3.12:

$ python3.12 badtrack.py
(1, 2)

But after the regression:

$ python3.14 badtrack.py
Exception ignored in: <function ResourceTracker.__del__ at 0x784fa13d0540>
Traceback (most recent call last):
  File "/usr/lib/python3.13/multiprocessing/resource_tracker.py", line 84, in __del__
  File "/usr/lib/python3.13/multiprocessing/resource_tracker.py", line 93, in _stop
  File "/usr/lib/python3.13/multiprocessing/resource_tracker.py", line 118, in _stop_locked
ChildProcessError: [Errno 10] No child processes
(1, 2)

Note that the script still ends “successfully”, but there is that ChildProcessError in the middle. The weird thing is that the error happens when closing the resource tracker, when it does waitpid(self._pid), being that _pid the process id of the tracker itself… it looks it finished before, so it’s gone at that moment.

This is 100% reproducible in my machine (Ubuntu 24.04, kernel 6.8.0-79-generic, x86_64 architecture).

A workaround is to call SharedMemory with track=False, but in some cases that is not possible (e.g. when using ShareableList).

Probably the best to do there is just wrap it with a try/except ChildProcessError: pass like the same code does in other corner.

CPython versions tested on:

3.13, 3.14, 3.15, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-140485: Catch ChildProcessError in multiprocessing resource tracker #141132
• [3.14] gh-140485: Catch ChildProcessError in multiprocessing resource tracker (GH-141132) #141384
• [3.13] gh-140485: Catch ChildProcessError in multiprocessing resource tracker (GH-141132) #141385

[bhubache]

I'm not super familiar with finalizers or the lower-level side of Python so apologies in advance if I'm spouting nonsense or this is generally unhelpful. With that said, once the process is forked, we should have the following process hierarchy:

─┬= 4278 ./python badtrack.py
 ├─── 4280 ./python badtrack.py
 └─── 4279  python -c from multiprocessing.resource_tracker import main;main(5)

When the forked process (4280) is getting cleaned up, ResourceTracker.__del__ gets invoked from within it. It calls waitpid(self._pid, 0), which raises a ChildProcessError because self._pid (4279) is not a child of the forked process. If this is an accurate assessment, should lines 114-122 be limited to execute only if the current process is the parent of self._pid?

[vstinner]

I wrote #141132 to fix this issue.