Skip to content

If a process fails during run_multiple_processes() we don't know about it until all processes have finished #44

Open

Description

@martinburchell

I've seen this with CRATE. If you launch a multi-process anonymisation and one of the processes fails, it doesn't get reported until all of the processes have completed and the die_on_failure argument has no effect.

The test files multiprocess_test.py and multiprocess.py demonstrate this.

With some debugging of subproc.py, it looks like TimeoutExpired never gets raised in _wait_for_processes().

except TimeoutExpired:

If I remove this call:

Pool(n).map(_print_lines, _g_processes) # in case of PIPE

the function works as I would expect. I don't know enough (yet) about the multiprocessing module to know why this is.

This is on Linux.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions