BioMoby

Eddie Kawas released a new implementation (no API changes) of the org.biomoby.client.CentralCachedCallsImpl class – now getting the contents of the registry using the RDF files. When all bugs are fixed, it should significantly speed up cache update in Dashboard and elsewhere.

Major changes in building and using jMoby were introduced during the BioHackathlon meeting in Japan. The changes, however, are not changes of the jMoby API – which means that your own code should continue to work without any changes.

The main issue is that jMoby uses now the 3rd-party libraries form the various Maven repositories. It fetches them when you compile jMoby, or when you use the new Ant’s task install. There is an article about how it is done (coming from a different project but using the same principles as in jMoby).

Another issue is that the Ant itself is not anymore distributed with jMoby. You need to install it separately on your machine.

API Changes:

1: Storage of xml-lang information in registry is necessary; providers should pay attention to which xml-lang attribute they send to the registry when they register their service/object/etc.
2: not all sub-elements need to be in an object instance. Document the fact that an element that is missing is different from an element that is blank.
3: explicitly say that member articles of an object are allowed to be child types of the registered type. We are now promoting a behaviour for member articles that already existed for the parent objects.
4: support Document style encoding as well as REST as well as the traditional RPC

TO DO’s:

1: Store all incoming registry data as UTF-8 MARK

2: Store the xml-lang of the incoming registry entry, and reproduce it on output from reg. MARK

3: support for async services in jMoby core MARTIN/INB

4: INB is going to propose using endpoint references as part of async (pass references to biiiig data) INB

5: everyone look at how to stream data in Web Services protocol EDDYONE

6: reorganize jMoby CVS so that tests are in a folder off of root. PAUL

7: Mark needs to update CommonSubs object parsing code to allow for missing sub-tags MARK

8: Mark needs to document about missing sub-tags MARK/EDDIE/WENDY

9: Others will look at their parsers to see if they need to be updated EVERYONE

10: Mark needs to update CommonSubs to allow sub-objects to inherit MARK/EDDIE/WENDY

11: Classpath.template should be added to CVS MARTIN

12: Martin is goign to change build process to “ant maven” MARTIN

13: Someone will commit to maven repository (need to get permissions on the existing open-bio maven repository) ALL JAVA DEVELOPERS

14: Java API documentation should either be reduced or shold be split into FULL and CORE – javadoc etc. MARTIN

15: WENDY needs to write an introduction to Moby document that explains what we mean by object, namespace, etc. and the basic behaviour that we are trying to achieve. WENDY

16: Someone (wendy?) needs to take charge of all documentation and FIX IT! Also try to get registry calls in some standard (BNF?) format. Registry API should be all in one document. Use XS3P schema documentation generator. WENDY

17: Martin makes a CPAN release of MoSeS + registry snapshot (one day) EDDIE/MARTIN

18: Mark will explore how feasible it is to register secondary outputs. Will make API change proposal to list. MARK

19: Releases of Moby Perl code should be in CPAN MARK WITH HELP FROM SOMEONE WHO KNOWS

20: Methods for testing services (at various levels) – including sample input/output in the LSID metadata. will become a part of jMoby’s API, and possibly a panel in Dashboard. NEED TO KNOW THE PREDICATE NAMES for this. MARTIN

21: Cleanly split Perl registry from java “stuff”. EDDIE

22: Andreas continues working on registry synchronization… fun fun fun! He will propose to the list, will take advice from community, and decide for himself what he wants to support. ANDREAS

23: Think about the proper way to attribute (e.g. logo’s) and state this in the documentation ANDREAS

24: MobyCentral needs to generate WSDL consistent with Document style encoding MARK/EDDIE

25: Moby Central needs to record the style that a service is provided -> add new fields to the registerService/findService input/output also the LSID metadata -> coordinate this with Feta MARK/EDDIE

26: Moby Central should somehow support registration of REST-style services. We decided that POST must be supported by any REST service. How does this affect WSDL? All REST services must be synchronous (at the moment) MARK/EDDIE

27: Ivan needs to look into support for Async services in REST (someday… manana) INB

28: Authentication -> how should we do it in the various libraries PAUL (Java)/EDDIE (Perl)

29: Paul will jMoby LSID library/API/implementation to get the metadata from service providers PAUL

30: Can we please make a WSDL/XML Schema file for Moby Central EDDIE

In attendance:

Mark, Eddie, Wendy, Richard, Ivan, Andreas, Paul, Martin, Mylah

We started with jokes about the danger of having the core Moby developers in the same room of a brick hospital built on top of a geological fault…

Issues that come up:
Atrtribution of data to providers
– set logo?

Should the Object ontology have multiple representations for any given concept?
– e.g. FASTA and DNA Sequence are redundant…
– we need to find a way to encourage object re-use
– we need a “rule-base” to use the SeaHawk transformation rules rather than registering a new Moby object; or do we still need to register a new object?

If we had a “referrer” that kept track of the previous service that was executed this would go into the service provider logs. We could then set up a service (pull) that Moby Central could use to gather this information from the logs. Alternately, we could “push” from the service as it is executed…

For us as client writers the first step is to ensure that we add the “referer” header… so we gotta do that! Probably the LSID of the previous service?

All in all, this will give us the ability to make the ‘next most popular choice’ very visible in our clients – filtering out the lesser-used services and bringing the cream to the top.

Problem #1: Registry

We have a problem that the registry is a mishmash of Perl and Java. The RESOURCES script is problematic since it is used by MOBY::Central to generate the RDF that is passed back from a registerService call. This needs to be Perlified. The agent is NOT a web script so that isn’t a problem. The isAlive “pinger” can be ported to Perl, but since it uses threads it will run slower in Perl than it did in Java. In any case, the entire registry can be written in Perl.

Decision: write a CPAN distribution
Decision: write an installer

Martin suggests “Perl Hacks” book as a must-read.

Problem #2: Services

need to support legacy. All 1300 services are currently RPC, so we have to keep that.

We need to add a field in the registry to indicate which style of encoding is being used. We prefer document-literal-wrapped.

Separate field in the database, API, LSID metadata, … field called “style”. Currently have a field called “category”, which has values “moby”, “moby-async”, and unofficially “post’. Separate “style” from “category”? Probably a good idea.

Tell Wendy that we need to put the async protocol up on the website.

IFF we provide “correct” WSDL will that make people happier? Maybe… we can provide WSDL in ~90% of the cases (though NOT for validation!)

Burning issue is that we don’t have XML Schema. Why? Because we allow more specialized objects to be passed into a service (and problems with HAS relationships). The world is moving to RDF, and RDF also cannot be expressed in XML schema, so the “political” arguments against Moby are going to eventually die-out.

One problem is that we pass a full XML document in our SOAP:body. If we use document encoding, we will need to strip the XML header which has other encoding info that we want to have… and the message will have to be parsed twice – once by the SOAP libraries and once by our Moby libraries (because we don’t have an XML schema to give to teh SOAP libraries to unmarshal the objects) and that will give us unacceptable overheads.

If we move to a REST style, then we don’t have these problems, but we don’t get any (what?) advantages of WSDL or SOAP.

We can use string encoding and send Moby messages exactly as we do now in teh body of a document-style SOAP message. No advantages of the SOAP libraries, but also no overhead of double-parsing of teh message (exactly as we do with the Moby XML-RPC style right now)

Conclusion: INB – could you consider asynchronous services support from a REST perspective and suggest a specification?

CONCLUSION: we will support RPC, Document-style and REST services, but REST services cannot (right now) be asynchronous.

Support for different languages

Registry contents should be UTF-8
– Registry code needs to check this, and if it isn’t UTF-8 then either convert it, or reject it, or save the encoding. DECISION: Registry converts all incoming XML into UTF-8.

From Services
– We send data in whatever encoding we want, and clients need to be clevererer… or parser (SAX/LibXML) already knows how to deal wtih these things. We need to pay more attention to this… that was the decision we made… to pay more attention.

Languages: if the xml-lang attribute is defined then it should be maintained and saved in the database and returned when the service description is reconstructed e.g. for a findService. need to capture the language for object, namespace, service, service instances, and secondary parameters.