Skip to end of metadata
Go to start of metadata
Implicit iteration occurs when connecting services with different depths, creating a new list of the outputs of the service.

Services output ports have a defined depth, just like service and workflow inputs.

Workflow output port depth

Workflow output ports have their depth calculated based on the connections in the workflow. This depth becomes the service output port depth when using the workflow as a nested workflow.

Inspecting List Behaviour

List behaviour is described below by inspecting the example workflow Pipelined List Iteration.

The first Beanshell script List_Emitter takes a single value count, and returns a list of depth 1 at the output port list. To inspect this, open the workflow, select the service List_Emitter, and go to Details->Beanshell service.

Now compare this with the definitions under Predicted behaviour, which should match the above.

If you inspect the workflow input port input, you should see that in this workflow it has been defined to carry a single value. In this case we've got a perfect match, the script expects a single value of depth 0, and that's what the workflow input port will provide.

Iteration Over Lists

If you inspect the next service in the workflow, Concat, you will find that it defines ports input and output, both of depth 0.

What happens when we connect the list for List_Emitter to the input port can be inspected by going to Predicted behaviour for Concat.

Lists to single value

If a service expects a list, but you provide it with a single value (like from the Text Constant), Taverna will simply wrap the value in single-item list(s) for you. Be careful, though, if you later change the workflow to do an implicit iteration over the service outputting a single value, the following service would receive the full iteration output in one go.

To workaround this, you can write a Beanshell script taking a single value, and returning a list with a single item.

This will force iteration over lists of lists below in the service below expecting lists.

What Taverna will do when running this workflow is to invoke Concat once for every element in the list at input, producing each element in the output list output. You can verify this by connecting output to a new workflow output port, and running the workflow. 

Taverna will however record provenance information for each individual run, so if you click for the intermediate values of Concat you should see the individual runs, listing what inputs gave which outputs, and when that run occurred.

Pipelining

While running you should also see a progress bar for each service in the workflow that is performing implicit iteration. 

You will also notice that services further down the workflow has started their iterations before Concat has finished its iteration. This is a Taverna featured called pipelining, which in some cases can significantly speed up workflow execution.

Depending on the relative speed of the downstream services, you might or might not see iterations queued up in the middle of the workflow while it is running. 

How to disable pipelining

If your services depend on external state in such a way that the other services should not start until the first service has finished all iterations (ie. a set of database inserts followed by a query for statistics over all added rows), add an additional control link by right clicking on Concatenate_two_strings and select Run after->Concat. Repeat for services that have to be run in isolation. Note that when iterating over lists of lists, the control link will not be activated 

Iterations Over Multiple Inputs

If you delete the boo string constant, and add a second workflow input as a list of depth 1, connect it to string2 on Concatenate_two_strings. Also connect output from this service to a new workflow output.

Inspecting the details of Concatenate_two_strings you should see that the service expects two single values, and return a single value, so all ports are at depth 0. This local worker takes two strings, and simply returns the string1 appended with string2

Looking at Predicted behaviour will however show that both string1 and string2 will receive a list of depth 1, and the output will now give a list of lists of values, depth 2.

This is because Taverna will iterate over the list at string1 - for each of those it will iterate over each of the values at string2 - effectively doing an all-to-all combination:

If a service with three single inputs were connected to three lists, the results would be a list of depth 3, list of lists of lists of values, as there would be 3 levels of iterations.

If an input expecting a single value is provided with a list of lists, that would make the result list 2 level deeper than connecting with a single value.

Similarly if a service expects a list (depth 1) and returns a single value (say number of items in the list), then feeding this service with a list of lists (depth 2) should give a list of values (depth 1).

It is perfectly possible for a service to only do iterations on some ports. So for instance a service with two inputs at depth 0, and one at depth 1, if given depth 1 lists to each port, no iteration would be performed over the third port.

You can change how Taverna performs iterations by configuring the service's list handling, for instance to decide which port is the "outer" iteration, or to change the iteration from all-to-all combinations to instead match each element on portA with the corresponding element on portB. 

Labels
  • None