Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Added section on tuning

...

If your workflow is likely to take longer than 24 hours, remember to increase its expiry time.

 

Tuning the Limits

Q: What are the implications of raising the limits “MaxRuns” and “OperatingLimit”? Are there maximum values?

A: The MaxRuns is the maximum number of workflow runs in any state that can exist at once. The OperatingLimit is the number of runs that are in the state Operating, where there's a workflow engine actually carrying out the workflow (as opposed to having you initialise things or collect the results); it should be lower than the MaxRuns, since it is a limit on a subset of the things limited by MaxRuns.

There are no hard-coded maximums, but that doesn't mean you should just raise the limits to super-high values.

The fundamental physical limit on MaxRuns is typically the amount of disk space you've got attached to the system (specifically, in the filesystem that is used for working space, which is usually /tmp). The amount of space required per run is highly dependent on the workflow being run; if you're making many copies of large files, it will obviously take more space per run than if you are just passing around short strings.

The usual physical limit on OperatingLimit is due to the amount of memory required to run the workflows, which is highly variable. (Each running workflow has at least one Java process to contain the execution engine, which may have many threads.) However, it is also quite possible to have workflows that consume a large amount of CPU, so that may also be a limiting factor. Since it depends quite strongly on what your typical workflow mix really is, it's hard to give definitive advice on how to tune.

Note that you can also tune the amount of memory that each workflow run is allocated via the administrative API, which allows setting various flags to be passed into the runtime.