Implicit iteration occurs when connecting services with different depths, creating a new list of the outputs of the service.
Services output ports have a defined depth, just like service and workflow inputs.
|Workflow output port depth|
Workflow output ports have their depth calculated based on the connections in the workflow. This depth becomes the service output port depth when using the workflow as a nested workflow.
Inspecting List Behaviour
List behaviour is described below by inspecting the example workflow Pipelined List Iteration.
The first Beanshell script
List_Emitter takes a single value
count, and returns a list of depth 1 at the output port
list. To inspect this, open the workflow, select the service
List_Emitter, and go to Details->Beanshell service.
Now compare this with the definitions under Predicted behaviour, which should match the above.
If you inspect the workflow input port
input, you should see that in this workflow it has been defined to carry a single value. In this case we've got a perfect match, the script expects a single value of depth 0, and that's what the workflow input port will provide.
Iteration Over Lists
If you inspect the next service in the workflow,
Concat, you will find that it defines ports
output, both of depth 0.
What happens when we connect the list for
List_Emitter to the
input port can be inspected by going to Predicted behaviour for
|Lists to single value|
If a service expects a list, but you provide it with a single value (like from the [string constant|taverna:String constant]), Taverna will simply wrap the value in single-item list(s) for you. Be careful, though, if you later change the workflow to do an implicit iteration over the service outputting a single value, the following service would receive the full iteration output in one go.
To workaround this, you can write a Beanshell script taking a single value, and returning a list with a single item.
This will force iteration over lists of lists below in the service below expecting lists.
What Taverna will do when running this workflow is to invoke
Concat once for every element in the list at
input, producing each element in the output list
output. You can verify this by connecting
output to a new workflow output port, and running the workflow.
Taverna will however record provenance information for each individual run, so if you click for the intermediate values of
Concat you should see the individual runs, listing what inputs gave which outputs, and when that run occurred.
While running you should also see a progress bar for each service in the workflow that is performing implicit iteration.
You will also notice that services further down the workflow has started their iterations before
Concat has finished its iteration. This is a Taverna featured called pipelining, which in some cases can significantly speed up workflow execution.
Depending on the relative speed of the downstream services, you might or might not see iterations queued up in the middle of the workflow while it is running.
|How to disable pipelining|
If your services depend on external state in such a way that the other services should not start until the first service has finished all iterations (ie. a set of database inserts followed by a query for statistics over all added rows), add an additional control link by right clicking on
Iterations Over Multiple Inputs
If you delete the
boo string constant, and add a second workflow input as a list of depth 1, connect it to
Concatenate_two_strings. Also connect
output from this service to a new workflow output.
Inspecting the details of
Concatenate_two_strings you should see that the service expects two single values, and return a single value, so all ports are at depth 0. This local worker takes two strings, and simply returns the
string1 appended with
Looking at Predicted behaviour will however show that both
string2 will receive a list of depth 1, and the
output will now give a list of lists of values, depth 2.
This is because Taverna will iterate over the list at
string1 - for each of those it will iterate over each of the values at
string2 - effectively doing an all-to-all combination:
If a service with three single inputs were connected to three lists, the results would be a list of depth 3, list of lists of lists of values, as there would be 3 levels of iterations.
If an input expecting a single value is provided with a list of lists, that would make the result list 2 level deeper than connecting with a single value.
Similarly if a service expects a list (depth 1) and returns a single value (say number of items in the list), then feeding this service with a list of lists (depth 2) should give a list of values (depth 1).
It is perfectly possible for a service to only do iterations on some ports. So for instance a service with two inputs at depth 0, and one at depth 1, if given depth 1 lists to each port, no iteration would be performed over the third port.
You can change how Taverna performs iterations by configuring the service's list handling, for instance to decide which port is the "outer" iteration, or to change the iteration from all-to-all combinations to instead match each element on portA with the corresponding element on portB.