table of contents
PROCESS_WORKER_POOL.PY(1) | User Commands | PROCESS_WORKER_POOL.PY(1) |
NAME¶
process_worker_pool.py - create a parsl process worker pool
DESCRIPTION¶
usage: process_worker_pool.py [-h] [-d] [-a ADDRESSES] --cert_dir CERT_DIR
- [-l LOGDIR] [-u UID] [-b BLOCK_ID]
- [-c CORES_PER_WORKER] [-m MEM_PER_WORKER] -t TASK_PORT [--max_workers_per_node MAX_WORKERS_PER_NODE] [-p PREFETCH_CAPACITY] [--hb_period HB_PERIOD] [--hb_threshold HB_THRESHOLD] [--drain_period DRAIN_PERIOD] [--address_probe_timeout ADDRESS_PROBE_TIMEOUT] [--poll POLL] -r RESULT_PORT --cpu-affinity CPU_AFFINITY [--available-accelerators [AVAILABLE_ACCELERATORS ...]] [--enable_mpi_mode] [--mpi-launcher {srun,aprun,mpiexec}]
options:¶
- -h, --help
- show this help message and exit
- -d, --debug
- Enable logging at DEBUG level
- -a ADDRESSES, --addresses ADDRESSES
- Comma separated list of addresses at which the interchange could be reached
- --cert_dir CERT_DIR
- Path to certificate directory.
- -l LOGDIR, --logdir LOGDIR
- Process worker pool log directory
- -u UID, --uid UID
- Unique identifier string for Manager
- -b BLOCK_ID, --block_id BLOCK_ID
- Block identifier for Manager
- -c CORES_PER_WORKER, --cores_per_worker CORES_PER_WORKER
- Number of cores assigned to each worker process. Default=1.0
- -m MEM_PER_WORKER, --mem_per_worker MEM_PER_WORKER
- GB of memory assigned to each worker process. Default=0, no assignment
- -t TASK_PORT, --task_port TASK_PORT
- REQUIRED: Task port for receiving tasks from the interchange
- --max_workers_per_node MAX_WORKERS_PER_NODE
- Caps the maximum workers that can be launched, default:infinity
- -p PREFETCH_CAPACITY, --prefetch_capacity PREFETCH_CAPACITY
- Number of tasks that can be prefetched to the manager. Default is 0.
- --hb_period HB_PERIOD
- Heartbeat period in seconds. Uses manager default unless set
- --hb_threshold HB_THRESHOLD
- Heartbeat threshold in seconds. Uses manager default unless set
- --drain_period DRAIN_PERIOD
- Drain this pool after specified number of seconds. By default, does not drain.
- --address_probe_timeout ADDRESS_PROBE_TIMEOUT
- Timeout to probe for viable address to interchange. Default: 30s
- --poll POLL
- Poll period used in milliseconds
- -r RESULT_PORT, --result_port RESULT_PORT
- REQUIRED: Result port for posting results to the interchange
- --cpu-affinity CPU_AFFINITY
- Whether/how workers should control CPU affinity.
- --available-accelerators [AVAILABLE_ACCELERATORS ...]
- Names of available accelerators, if not given assumed to be zero accelerators available
- --enable_mpi_mode
- Enable MPI mode
- --mpi-launcher {srun,aprun,mpiexec}
- MPI launcher to use iff enable_mpi_mode=true
May 2024 | process_worker_pool.py 2024.05.06+ds |