Skip to end of metadata
Go to start of metadata

This is part of the tutorial Tutorial - Service invocation plugin

Service invocation

Now having configured the activity, it is possible to add it to a workflow and connect up the ports, as we did in step 2. Testing the service in Taverna. At execution time, data will be available at the input ports, and the activity is responsible for performing the intended action, and return references to values on the declared output ports.

If we scroll down to the executeAsync() method, we can see how the actual service invocation takes place.

First thing you will notice is that the executeAsync method does not directly invoke the service. The actual invocation is inside a Runnable, which is requested to be run using the callback object. This asserts that the execution does not block other executions in the workflow, as the callback will be responsible for allocating a thread from the thread pool and start the execution. Results are also returned using the callback object, as {{Runnable}}s can't return anything.

Support for asynchronous services

This asynchronous invocation gives the activity implementation flexibility to handle several request in a single thread (or using it's own thread pool). If the activity is invoking an asynchronous services with a typical upload-submit-status-status-status-retrieve cycle, the activity could have a single running thread that check the status of all submitted jobs in order, and when results are ready for a particular job, it can pick up the matching callback object and return the retrieved values, thus potentially handling thousands of concurrent service calls using a single thread.

Resolving inputs

The first thing done inside the run() method is to resolve the input port values.

This chunk of code retrieves the ReferenceService from the InvocationContext. The reason for this is that the parameter inputs come as a Map<String, T2Reference> -- that is a map from input port name to a data reference. As in this activity we want the real String behind the input port "firstInput", we need to resolve this reference. The renderIdentifier method takes three arguments, the T2Reference to resolve, the desired class of the output, and the InvocationContext. This class will typically match what we stated in the configurePorts() method, although this is not a strict requirement.

In this example we have made the activity be configured with an additional input port "extraData" which is conditionally added depending on the activity configuration. More on the configuration further down.

In this chunk we are (optionally) resolving the second input port "extraData"". The data here is a bit more interesting. First of all the data is retrieved binary as byte[], and not as Strings. Internally strings are always registered as utf8-encoded bytes, but it is also possible to register binary byte arrays directly, for instance for PNG images -- however these can't be rendered as a String.

In addition the reference is to a List of such byte arrays, as we defined this input port as having depth 1.

Because of this we know that we declared the port as depth 1, and so it's safe to cast to a List<byte[]> instead of a byte[] if the depth was 0. You could still check the depth of a reference by adding these lines:

We'll try this now. Save the modified ExampleActivity, check in Eclipse that Project->Build automatically is enabled. If your Developer Taverna Workbench is still running, quit it. Then click and hold on the Debug button in the Eclipse toolbar, select TavernaWorkbenchWithExamplePlugin. The Debug mode should allow us to inspect and slightly modify the activity code while the test workbench is running.

We'll rebuild the workflow (but without the output ports). To trigger the extra input port after adding the service to the workflow, we'll have to modify the configuration bean. Luckily this example comes with a user interface for changing the activity. First select the Examples_1 service, Right click and Show details. Then under the Details tab, click Configure.

In the service configuration dialogue, set specialCase as the Example string.

After applying and closing, if you now click to view all ports of the service, you should see the additional ports extraData and report.

Add the local worker Create lots of Strings and connect it to extraData. Set a string constant on firstInput. You don't need to create the workflow output ports, as we are no longer interested in the output from the service. Now run the workflow.

In Eclipse under the Console tab (Window -> Show view -> Console), you should now see that the reference is of depth 1:

Taverna 2.4

In Taverna 2.4 the console output is automatically redirected to a log file, so will not appear in the Eclipse console by default. You can change this by selecting File -> Preferences -> Workbench and ensuring that the option "Capture output on stdout/stderr to log file" is not ticked.

More information on preferences in Taverna Workbench can be found in the User Manual.

Put a break point on the System.out line by selecting it and selecting Run -> Toggle Line Breakpoint. Run the workflow again from the running developer workbench. Eclipse should switch to the debug perspective and pause at the System.out line.

In Eclipse, click Window -> Show view -> Display. Type into the Display window:

Then highlight the text and click the little on the toolbar for Display results of evaluating text.

As you see, we can render the same identifier in different forms, in this case both as byte[] and String. This reference can be dereferenced as a String because that's how the Create_lots_of_String service registered them.

Performing the work - invoking the actual service

Now to the piece of code that is missing - the actual calculations, where we should call our underlying libraries or external services.

We'll replace this code with something that actually works. How about an SHA1 checksum of the binary data? Add this snippet after the dummy invocation code.

This should calculate a checksum of all the concatenated byte arrays from the list special.

Notice the callback.fail() call, this would fail this invocation instead of returning data. An error document based on the fail message and exception will be registered and passed to the next services in the workflow (who would then typically skip this item). It is important in this case to also return immediately, so that we don't by mistake also do a callback.receiveResults() later. In the case of runtime exceptions, if say a NullPointerException occurs, then the callback has installed an error handler on the invocation thread that will take care of doing callback.fail() for you - but you would not be able to pass a human readable message in that case.

Adding dependencies

We'll need to add a Maven dependency to find that commons-codec class Hex. This Apache commons library will be our example of how to add an external dependency.

An older version of commons-codec is inherited through our activity-test-utils dependency - but this is only used by the unit test. That version does not seem to have the encodeHexString() method, so we'll declare our activity to depend on a newer version. Right click on the myfancytool-activity project, and select Maven -> Add dependency. Type commons-codec into the field, select version 1.4 - commons-codecs-1.4 (not -javadocs or -sources) and click OK.

If you do not see any artifacts listed, it could be because you have not installed the m2eclipse plugin Maven Central repository index (optional). If this is the case, instead open the pom.xml from the myfancytool-activity project, and go to the Dependencies tab, where you can add a dependency with group id commons-codec, artifact id commons-codec and version 1.4.

Adding your own libraries

As Taverna's plugin system is based on Maven and Maven repositories, you need to state dependencies on other libraries (JARs) using Maven.Most open source libraries like jdom and log4j are already available in central or other public Maven repositories, use search engines like http://repository.sonatype.org/ and http://mvnrepository.com/ to find such artifacts. If you need to depend on any of your own JARs, you need to first make them available in a Maven repository. You can use the Maven install to install such JARs locally to your .m2/repository folder while developing, and the Maven deployplugin to install these JARs when you want to make your plugin installable from within Taverna.

After saving the file you should be able to find commons-codec in the Maven dependencies classpath folder. Use Navigate -> Open type and Hex - it should find Hex - org.apache.commons.codec.binary - C:\Users.... Press OK to open the class. Make sure in Package Explorer that Link with editor is enabled. You can also right click on the commons-codec-1.4.jar entry and select Maven -> Download sources - this would download source code for the JARs where available. All Taverna dependencies should have such source code, making it easier for you to inspect Taverna code without having to check out the full source tree.

If you want all the relevant Taverna source code available in Eclipse, right click on the myfancytool-activity-ui project and select Maven -> Download Sources there - this would then also include the related workbench source code.

Returning data

Finally, back to our service implementation class ExampleActivity. First let's check that it still runs with our new code. Right click on the ExampleActivityTest class (from src/test/java) and select Run as -> Junit test. As we've not fundamentally changed what the example activity is doing, the tests should all pass.

Let's have a look at the code for returning data. Taverna refers to data by references, this allows Taverna to store larger data in a local database to avoid filling up memory. For instance if your workflow takes 10 minutes to run, processing large images one at a time, Taverna can store the unprocessed images in a database until it is needed by the next service in the workflow. (Note that you need to disable In-memory storage under File->Preferences->Data and provenance for this to be apply.)

So when an activity has received some output data from the real service implementation (in this case our calculated checksum), it will need to register those outputs with the reference service, which will return {{T2Reference}}s that can be returned using the callback object.

This example registers a string simple (which is at depth 0) and put it under the simpleOutput port name. Let's replace that string with our hexified checkSum:

You can also register a whole list of values at once:

The reference service will take care of registering a T2Reference for each of the values in the list, and then registering a list of those references as well. The returned reference is therefore a reference to a list of depth 1, and the items of that list are references to the values "Value 1" and "Value 2". This list reference can therefore be returned for moreOutputs that we declared had a depth 1 in configurePorts().

You are always required to specify the depth of the list that you are registering, it will typically match the depth of the output port. The registration would fail if your list is of the wrong depth, for instance if you tried the above with depth 2, or you tried to register a list with leaf nodes at inconsistent depths. (Say a list (depth 2) where the first element is a list (d=1) of Strings (d=0), and the second element is a String (d=0)). Even empty lists require a depth, to enforce this list consistency requirement. The reason Taverna is strict about this is that such lists are used in implicit iterations, which would not expect a String in the middle of a lists of lists.

This last output port is included to remind us that we need to return data for all output ports at the same time, so if we declared the output port report, we'll also need to return an output for this.

In non-critical cases where you are not able to return data for all ports, you can register an Exception instead of a value for the failed port - this will be converted to an error document instead. Let's try that instead of our list on moreOutputs. Replace the code from moreValues = ... till outputs.put(OUT_MORE_OUTPUTS... with:

Notice that this error document will be of depth 1 - as it is in place of the expected list. That means that any downstream services connected to moreOutputs that are expecting single values will be able to "iterate" over the error document, as a derived error document at depth 0 comes out of the iteration.

Let's complete the changes and also change report = "Everything OK"... with a proper report:

Finally, let's check and update our unit tests to match our new activity behaviour.


This is part of the tutorial Tutorial - Service invocation plugin

Labels