The aim is for the work outcomes to be useful for myGrid (naturally) and for the work described to be challenging but tractable for David.
We are working out some of the details of the work areas as we go along, with the focus being usefulness and tractablility.
In agile terms we are preferring collaborations over contracts; possible due to the strong trust between the team and David.
Please read the original Taverna 3 contract page for context
A stable Taverna 3 release is the target for this contract.
Contract to start 23th of September 2013 to 20th of December 2013.
M5 - Sept/Oct 2013
M6 - Oct/Nov 2013
M7 - Nov/Dec 2013
- JIRA for tasks (all major work areas detailed in this document) - (from 02/09 meeting this needs to be re-emphasised)
- David to send summary of work done on Friday 5pm
- David to come in for face to face meetings on Mondays (usually) (02/09 - this was fine)
Things which will not be part of the Taverna 3 contract
Features in Taverna 2.x which will not make it to Taverna 3 release
- Custom perspective i.e. user layout
- Taverna 1 compatibility; Taverna 3 won’t be able to load Taverna 1 workflows
- No BioMoby support (we might need to ask users about this)
- big job
- requires time (02/09 - myExperiment and support show no high usage of BioMoby)
- is it worth the time
- not impossible to do
prerequisite myGrid work
These things need addressing before Taverna 3 can be delivered :
- Data Bundle specification/API
- progress made in contract 1
- need a release version
- Format for saving workflow runs (needs a discussion between David (run report requirements needed by saving step) and Stian) Taverna 2.x version ready by end of Sept 2013
- Strategy for preservation of old CVS/SVN history when moving to GIT (02/09 - Donal to take the lead on this done by time we release software - can be done incrementally) (23/09 - David says more urgent because other are working on the code)
- restructure and move or
- move and restructure
- what are the pro's and cons
- and who will do the actual move - David ? Donal ?
- hard part: Agreed module structure by the team (02/09 Donal (leader), Stian, Alan (advice))
- Provenance fit for purpose to capture information needed for SCAPE/BioVeL (02/09 - by end of Sept in good time for BioVeL plenary)
- save the provenance as well as the date from a run (David)
- do we need a workflow run activity as part of Taverna 3 so you can query provenance, monitoring data and data outputs as part of a workflow (need a discussion) (02/09 - probably not needed by Dec release can come as a plugin later (Donal, Stian and Alan in 02/09 meeting))
- re-engineering the provenance system to be extensible for each activity type - David to write up his ideas - this is 2-6 weeks work and has a lot of risk associated with it so perhaps not for Taverna 3.0.0 (02/09 - need to revisit this with David around - might need to do some sooner and 2-6 weeks to implement ? or document ?)
Source code layout (Donal)
- Move to git - agreed Taverna 3 move to git - team to discuss level of history kept
- Move to github - agreed
Forward port of Taverna 2.x changes since 2.4.0 to Taverna 3.x (new section 02/09)
- Discussion around Mark Borkum plugins (JSON, REST, OAuth) - myGrid to do work with support from David (02/09 Alan to own this area (maybe do the work post discussion))
- Critical to get in components and interactions (blocking) - Will need David to work on components, interactions done by myGrid) (23/09 - it's actually Stian and Alan as test of plugin)
- Dmitry WSDL parser - needs a discussion. (02/09 need to find out if it works with OSGi - Stian to own this one - needs a discussion when David is around) - put in a list of things to discuss when David's here to see if it is in scope - good task for learning the framework (not a blocker) - would update mechanism support this ?
- (02/09 - David to get the build working as soon as he starts so we can make it a baseline) - needs something runnable that jenkins build and for cmd tool can test as part of CI
- New module structure (draft - to be proposed by David W)
- Core (engine)
- Command Line
- Server (02/09 Donal - build structure should follow code structure)
- Report on coding style guide usage as part of build process (02/09 - myGrid team does not feel this is critical path, maybe useful for Foundation - task to be done by David) (23/09 - SemanticVersioning check package that Stian found)
- Copy and paste detection (02/09 - don't develop this by scratch - if there is something available then this can be used by David) (23/09 - pmd plugin from Maven) (23/09 - David said also not checking on impl - it's just API's)
(23/09 upto half time - HERE)
A strong overall aim of this work is to enable a true continuous release mechanisms for Taverna. Most of the work will be hardening, documentation and porting (and any necessary bug fixes) as oppose to features.
- Documented process/routine for doing an update
- How to do a quick patch? (“Service X needs new parameter Y!”, “Snow leoapard graphviz does not work!”)
- How to do a bigger update (consistening of a set of updates) - current configuration -> new configuration : analyze to determine osgi bundle changes.
- say several APIs have changed in platform and we need new command line options and a new button in the Run dialogue
- How to do a new download?
- How can it be picked up by old installations? Do they need to reinstall? (That should be OK as long as they are not left behind) - meeting Stian, Alan, David needed
- Effect on plugin developers? - meeting discussion and decide what to do - e.g. API need to be strictly extensive - can we live with this - discussion David and Donal
- Plugins compatibility check against updates
- Plugin support (archetype, deployment)
- Remove any remaining impl dependencies (may require new api/impl splits)
- Porting of existing plugins
- Port myExperiment (as is) - Shoaib to talk to David about this - define the API/IMPL split - refer to workflows repository - maybe AstroTaverna with Stian.
- Update architype for activity code For David
- Supported in tooling - Maven archetype to help make project plugins
NOTE - report need at 6pm on Friday - get him to e-mail the Jira things he has been working on, note things he realised need to be worked that he is not working on.
- Make modules more OSGi aware - services transient and not all there at the beginning - ACTION - ask David if this is done to a good enough level
- Implement other service types
- need to go through service types with Alan and see what needs to be done in context (e.g. tool service) -
- Pickup config and preferences in one way (OSGi config service) - ask David if the way he has done it as a service - in which case no need to do the OSGi config - ask him how cmd line will deal with proxies
- Kymbat usability changes, fairly simple low hanging fruit ones - ACTION - Alan to go through with David to see what has been done and what has not
- improve iteration strategy on diagram - this is a must for c2 - get this working with the latest working of Dot (action in itself) - Taverna's dot parser - 2 weeks work 1 week test - sounds important - (donal used leopard version on mountain lion and it worked - not ideal)
- Scope out change needed for big data (e.g. reference service and the db backend)
- more of a documentation task - guidance of existing system as well as how to impove the system to handle bigger things
- need a definiton (more than one disk, more than in memory, partitioning) - one of the team get David input - Donal to lead
Documentation (this looks like a month of work) documentation and fixing bugs
- Coding style guide
- Testing standards guide
- Developers guide - (include osgi bundles, dependencies guide and versioning rules)
- Plugin developers guide
- Plugin migration guide from 2.x to 3.x
Test top 20 workflows on myExperiment (none BioMoby ones)
<Alan to provide specification> of testing (e.g. load, run, provenance) - all of the myExperiment workflows which is used for release - point David to them Alan
Any new and modified code should be covered by appropriate unit test code coverage.
Report on test coverage to be provided.
MyGrid work (handover and post delivery)
Not an exhaustive list; but indicative of the split of work
- add interaction service (this should be at the start of M5) - Alan to do - ideally in M7 - but could be after beta - depends on workload
- port & refactor myExperiment plugin - after the beta
- port & refactor BioCat plugin - after beta - Alan says needs a big re-design
- Add component plugin (starting from existing 2.x Component plugin) - Alan and Stian to do - but after the beta - however - provenance is more important for SCAPE and BioVeL
- write user guide to Taverna 3 - Alex N - do before beta - immediate pre-release
- Update archetype for activity tutorials - Alex N - with the beta - or could come a month after
- BioMart 0.8 in the Taverna 3 work - Donal - after beta
- Verticals - Stian - take out defaults, look after branding, decide release or config - after beta
- renderers discussion (keep or not) - early on in M5 (starting on 20/sept)
- Update the server to work with the new Taverna 3 command line - Donal
- Testing on alternative exec environments e.g. running on a server - Donal - Shoaib's think with or soon after beta
- Integration of Dmitry firstname.lastname@example.org updated WSDL parser - after the beta - Stian - ideally for the release
- (UPTO HERE ON 20/09 Taverna planning meeting)
- Improved XML handling - replace splitters with wsdl activity configuration via via XML tree representation (SCAPE requirements - helps wsdl to local service transition) - really needs a new grant proposal
- improve iteration strategy dialog
- Data output structure/size predictor (e.g. put in input data and see the effect on size/structure of output data) (least priority)
- Error requirements (Alan) and then David to apply to beanshell/script service and the UI
- Taverna 3 workbench
- Taverna 3 platform service (OSGi)
- Taverna 3 platform jar for none OSGi environments
- Taverna 3 commandline
- Taverna 3 server with alternative execution environment support (e.g. WB running a workflow on a server instance)
- needs to update the server (use the updated command line in the first instance rather than the platform)