Code freeze of Taverna 1.7.1

We’re hopefully doing our code freeze for Taverna 1.7.1 today, 2008-03-28.

There’s a few bug fixes, in addition to quite a bit of work on t2. There will also be a drag-and-drop-able, interactive workflow editor, basically you can edit the workflow by connecting lines between the processors. Those of you who remember Taverna 1.4 might notice that this is something we’ve had hidden in the shelves for a bit, we hope that as we bring it back this time it will be a bit more usable as an alternative way to build workflows. You will still be able to flip to the “Graphical” tab for the classic non-interactive (but usually prettier) diagram.

The t2 plugin will now be a bit more usable with support for BioMoby, we’ve also made the BioMart support use streaming so that as soon as the first row of the BioMart result set has been received, it will immediately be pushed down to the next step in the workflow. Further on, the results of those operations will also be streamed on immediately before the full list is processed. For large datasets this should significantly improve the execution time of the workflow.

The t2 plugin also now has a graphical representation of the workflow as it’s running, with progress bars. Note that when doing streaming the progress bar can look a bit weird as results are coming in – say initially processor 2 has finished 4 of 5 items, but 5 new ones comes in from above, then the progress bar will jump from 80% (4/5) to 40% (4/10). But the cool thing is that you will be able to almost see the data as it’s flowing through, almost like pipes with pumps between them. We’re planning to add more features to this view, say to let you click on a processor as it’s running and have a look, perhaps tweak some parameters or go deeper in detail.

After the 1.7.1 release, which we’re predicting will be out in about a week (2008-04-07), we’ll focus fully on the 2.0 release for June 2008. The 2.0 release will feature the t2 enactor as the core engine, and some graphical updates as well. We’ll try to post more on the progress for 2.0 here as we go along.

Future visions

Writing a User Interface is quite often the hardest part of any development job. Everyone likes different look and feel and they seem to take forever to develop. I tend to hand code using Swing (Java’s default GUI) because I have not found an interactive GUI designer which I am comfortable with. In the (not so) distant past I used Delphi which had a really nice, intuitive GUI builder and I have also played around with the OSX development tools so why doesn’t Java have something as good. Maybe the old style application is dead and browser embedded GUIs are the way to go? The Nintendo Wii and the iPhone have shown what you can do with some lateral thinking (and a large budget). Check out http://www.cs.cmu.edu/~johnny/projects/wii/
and http://www.youtube.com/watch?v=0awjPUkBXOU&feature=related for a vision of the future.

The Taverna 2 (T2) development continues with the initial release planned for late June. Although most of the team are collocated on one site (Manchester University, UK), Taverna is an open source project. However, we have not had much “external” development for a while and this is something we will address with T2. The plan is for both user and developer forums nearer the time (real and virtual (maybe via irc)) and to promote T2 and it’s open source ideals at all the conferences etc. that we attend. One of the problems we face is having a governance regime which can handle external contributions in a controlled manner. There are lots of successful open source projects out there and they must have solved this problem. You can look forward to a “Vote for this feature” on the new look Taverna/myGrid website soon(-ish)!

Also, OMII-UK – which Taverna is part of – has been accepted as a mentoring organisation in this year’s Google Summer of Code. Maybe you would like to develop some cool code for us and the open source community!!

Where in the world?

A recent project involved the use of semantic metadata, stored using RDF (http://en.wikipedia.org/wiki/Resource_Description_Framework) and used mainly on a Globus based grid (http://www.globus.org). Inside the RDF was locational information which consisted of a place name, person, date and event. Using the Google maps API I took the RDF, extracted the locational info using a SPARQL query (http://en.wikipedia.org/wiki/SPARQL) the Sesame rdf tool (www.openrdf.org) and geo-located in a map.

Here is a sample of the RDF:

<j.0:Topic>Semantic Grid</j.0:Topic>
<j.0:Place>Ischia, Italy</j.0:Place>
<j.0:is_given_by rdf:resource=”http://www.ontogrid.net/ontogrid/sew/rdfTerm?

var_name=ontology&ontology=OntoGrid+Person+Ontology#A_Person”/>
<j.0:Scope>Academic and industrial</j.0:Scope>
<j.0:is_given_by rdf:resource=”http://www.ontogrid.net/ontogrid/sew/rdfTerm?

var_name=ontology&ontology=OntoGrid+Person+Ontology#Another_Person”/>
<j.0:Start_Date>09/07/2006</j.0:Start_Date>
<rdf:type rdf:resource=”http://www.ontogrid.net/ontogrid/RDFS/OntoGrid_Event_Ontology.rdfs#

Course”/>

I’m not sure SPARQL is quite ready for the mass market yet, can we really expect people to write things like this when they are browsing their semantic web?

query = “PREFIX loc:<http://www.ontogrid.net/ontogrid/RDFS/OntoGrid_Event_Ontology.rdfs#> ” + “\n” +
“SELECT ?x ?name ?date ?place \n” +
“WHERE{?x loc:Name ?name \n” +
“.?x loc:Start_Date ?date \n” +
“.?x loc:Place ?place}”;
Anyway, the code was JSP since Sesame is a Java API with Javascript bits for the map part. The hardest part to figure out was how to loop over a Java array and populate a Javascript one. Lots of head scratching and I came up with this:

<% for (int i=0;i<events.size();i++) {
%>
info[<%= i %>] = “<%= events.get(i) %>”;
<% }

%>

The info array is the javascript one and the events array is Java/JSP.

The geo-location part of the Google Maps API was another cool bit, all you have to do is pass it in a string with the place name and it will do it’s best to return a lat/long which you can then place on a map using a marker. The marker itself can be clicked on and pops up the info about the event at that location

geocoder = new GClientGeocoder();

geocoder.setBaseCountryCode(“GB”);

function getAddress(address, date, type) {
geocoder.getLatLng(address, function (point) {
if (point) {
var marker = new GMarker(point);
map.addOverlay(createMarker (point, address, date, type));
}
}
);

}
function createMarker (point, address, date, type) {
var marker = new GMarker(point);
GEvent.addListener(marker, “click”, function() {
marker.openInfoWindowHtml(“<B>” + type + “</B><BR>” + address+”<BR>” + date);
});
return marker;
}

After that I had a fully functioning, clickable google map which everyone in the project could browse to see who had been where and why (useful for showing how important your project is!).

Going round in circles

I started this blog mainly as a dumping ground for ideas and things I have learned so that I didn’t forget. Like a virtual notebook. However I thought it would be quite cool to access it from anywhere but I just couldn’t get onto my site from anywhere on the web. Just kept getting a 301 http error. I tried lots of suggestions, everything pointed to mysql, Leopard & Apache 2 not quite co-existing nicely. I tried things like changing the mysql.sock location but with no success. In the end I saw a bit of advice on (of all places) the wordpress forum which suggested changing the blog locations on the wordpress options page.

http://wordpress.org/support/topic/142498

Unbelievably it worked. Now, I don’t know if the mysql settings needed changing as well but everything is sweet now.

I also had some weird problems with Adium (a chat client for OSX) at the same time as the 10.5.2 and latest Adium updates.  It would disconnect from jabber constantly.  Strangely all I had to do was start with a fresh set of preferences – http://trac.adiumx.com/wiki/Preferences

Point of View

Taverna can display results based on their MIME type eg. Text/Html. That means that T2 will have to do the same. Most of the renderers could be ported across from their T1 equivalents but a few had to be redeveloped due to changes in the data management structure fot T2. Results are now passed by reference and can be “de-referenced” into whatever type you request as long as the Data Manager will allow it. The results can then be viewed by the MIME types associated with them. To allow easy extensibility (or “extension points”) Taverna 1 & 2 use Java Interfaces a lot. These extensions are implemented using the Service Provider Interface (SPI) pattern. This is not a common one and there is not a great deal of info on the web about it. In the Renderers case, anything that wants to be a Renderer must implement the Renderer interface. To find the renderers they must also be registered by writing their full class name (including package) in a file stored in renderers-impl/src/main/resources/META-INF/services/net.sf.taverna.t2.renderers (as you might have guessed, net.sf.taverna.t2.renderers is the package name). The SPI lookup is performed by having a RendererRegistry which does something along the lines of getClass().getClassLoader.getResourceAsStream(“/services/net.sf.taverna.t2.renderers”). The individual class names can then be found from this stream and additional checks (defined in the Renderer interface) can be made on them to see if they support the Mime types you want.

The Installation blues

Two of our users had some problems with R (the stats package) and the server component Rserve. So, figured I should install it on the Mac. Downloaded the binary package from http://cran.r-project.org/bin/macosx/ .All was going well until the package tried to install tcl/tk at which point it failed with a strange message about “could not install some files to /”. Not very helpful. R was OK though but all attempts to get Rserve failed since it needed the tcl/tk stuff (ps. OSX Leopard has X11 be default now) maybe a source install was the best idea. Downloaded the tcl/tk source from http://www.tcl.tk/ and followed the instructions on http://www.tcl.tk/doc/howto/compile.html#unix .Some of the tests failed though and I had no enthusiasm for trying to figure out why (although sunning as sudo helped solve fCmd-9.4 file rename: comprehensive: dir to new name FAILED). The other one was more obscure

==== unixFile-1.7 Tcl_FindExecutable FAILED
==== Contents of test case:

set env(PATH) “:/dummy”
testfindexecutable junk

—- Result was:
/Users/Ian/Downloads/tcl8.5.1/unix/junk
—- Result should have been (exact matching):
/Users/Ian/downloads/tcl8.5.1/unix/junk
==== unixFile-1.7 FAILED

Why it should be changing “d” to “D” I have no idea. Anyway, I moved the whole package to my root but still got a failure with

==== exec-9.7 commands returning errors FAILED
==== Contents of test case:

list [catch {exec [interpreter] “$path(sh)” -c “\”$path(echo)\” error msg 1>&2″ | [interpreter] “$path(sh)” -c “\”$path(echo)\” error msg 1>&2″} msg] $msg

—- Result was:
1 {error msg}
—- Result should have been (exact matching):
1 {error msg
error msg}
==== exec-9.7 FAILED

Lost enthusiasm completely by this point and googling didn’t help too much but it gave me the idea (from http://ircarc.com/freenode-tcl/302.html) that perhaps the tcl8.5.1 source wasn’t the way to go. Downloaded 8.4.18 and everyone went well until I got cp: /usr/local/lib/#inst.17465#: Too many levels of symbolic links

For whatever reason /us/local/lib was symlinked to itself. So, I removed that and everthing went well (except for a few tk test errors but I suppose that is to be expected – kind of cool watching all the X11 windows appear and flying around on their own!!).

So, now to install Rserve. The only way I could get it to work was to download Rserve_0.5-0.tar.gz and type sudo R CMD INSTALL Rserve_0.5-0.tar.gz

Sweet, everything sorted. Now, go into R. Type library(Rserve) then Rserve() and there it is running on port 6311.

Nice. I need a rest now (and I haven’t even looked at the bug yet!!)

Update – The Rserve version I installed was too new and uses a different protocol from the Rserve clients in Taverna. Oh well, time to give up I think!

Big cat attack

Hello, I am a developer on the Taverna Workbench project for Computer Science at Manchester University in the UK. This is a workflow engine for consuming and running ‘everything’ (web services etc. etc.). It is primarily used by Bio-Informaticians but is also used by Geologists, Chemists, Musicians(!) and more. It is written in Java and developed using Eclipse and Maven and as you might guess this has it’s fair share of problems, solutions and exciting revelations. The problem is that we all forget how we solved last weeks problems so it’s probably a good idea we shared them with everyone else.

Anyway, it’s been about 6 weeks with Leopard on my iMac. So far, so good. No major issues. Eclipse clashed with spaces key settings which meant a change to the eclipse keys. The 3d dock is fairly pointless and ditto for the see through toolbars. PHP was easy to set up once I figured out that Apache now lives in /etc/apache2. I even got wordpress working by using 127.0.0.1 instead of localhost for mysql.

However, been having a few Maven/Eclipse issues, things like it will build on the command line but not in Eclipse etc. Probably nothing to do with OSX so I figured it was time for a re-install of Eclipse.

Rule 1 – remember the Java Build Path/Libraries trick for Maven ie. enable dependency management then double click on the Maven libraries in the build path and select ‘Include dependencies and and source folders from modules’. ‘Update Source Folders’ should then work.

Rule 2 – give the eclipse JVM enough RAM to avoid the ‘run out of heap space’ problem. Ctrl/click on the Eclipse App icon, show folder contents and modify the Contents/MacOS/eclipse.ini file. Change -Xmx256m to -Xmx512m and -XX:MaxPermSize=256m to -XX:MaxPermSize=512m (or whatever size you need,I’ve only got a paltry 1GB, should upgrade to 2GB some time).

Oh, another thing. I always forget the settings to run Taverna and new plugins within Eclipse. So, just for the record:

Rule 3 – Remember your ‘Run As’ settings, try something like

-Draven.eclipse=true
-Dtaverna.dotlocation=/Applications/Taverna.app/Contents/MacOS/dot
-Dtaverna.startup=/Users/you/workspace/taverna1.0/src/main/resources/workbench
-Dtaverna.home=/Users/you/.taverna-snapshot -Xmx512m
-Draven.repository.0=file:///Users/you/.m2/repository/

But there is one big complaint about OSX10.5 Leopard – where is Java 6 ?! By the time Apple release it for the Mac everyone else will be using 7. This is becoming a bit of an issue for us. I don’t think Apple can claim that they are big supporters of Java anymore until they sort out their act and get 6 out the door. Beta versions don’t count (and fairly screwed up my install of Tiger). Do people really care about Aqua bindings for Ruby and Python? Come on Apple, give us Java 6 for OSX NOW!!

The developers of myGrid tell of their quest of the code