Run Commands
Read the PPS series >

Parallelism Spec PPS

Define the number of workers used in parallel.

April 4, 2024

Spec #

This is a top-level attribute of the pipeline spec.

{
  "pipeline": {...},
  "transform": {...},
  "parallelism_spec": {
    "constant": int
  },
  ...
}

Behavior #

HPE ML Data Management starts the number of workers that you specify. For example, set "constant":10 to use 10 workers.

  • The default value is 1

When to Use #

⚠️

Because spouts and services are designed to be single instances, do not modify the default parallism_spec value for these pipelines.