Tag Archives: provenance

Stand up and be counted

Taverna workflows are full of shims. That’s a fact. Shims are the little adapter services, mostly using Beanshells, which convert the outputs of one workflow processor before sending it to the input of another. They are always being re-invented and 90% of the time do the same things – concatentate strings, swap things around…..

The problem is that these shims are designed to be use once, almost throw away, so they are not annotated with any sort of helpful information. This becomes a problem with the scientists greatest challenge – provenance and data lineage (OK, I might be being a bit melodramatic about it being the greatest challenge but it is up there somewhere – maybe nearer to the challenge of making the perfect cup of tea, no mean feat). Data goes into a shim and comes out the other side but what is happening inside? Shims are not the only processors guilty of this, there are plenty of black box services out there in the world. So, how can we address this problem? Well, we have started to collect the shims that people actually use in a my experiment group, we will then figure out all the similarities and come up with an annotated set which we can all use from Taverna 2 (T2). The current idea is for the T2 workbench to have an intelligent workflow designer which will recognise that you are trying to do some shim magic and suggest one to use. Maybe we will need a Taverna clippy style pop up (think Word etc) – ideas on a postcard……

So, if you are a shim it’s time to stand up and announce to the world – “I’m a shim and I’m proud of it”.