- RESTful services
- WSDL/SOAP Web services
- Service creation and deployment guidelines
- Design the service
- Consider how a service consumer will use the service
- Design your services with compatibility and interoperability in mind
- Consider reliability and stability of your services
- Consider how registries and consumers can monitor the service
- Do not "press a button" on existing code.
- Allow the service to be used by consumers who know minimal information about the service
- Create services that do what consumers want to do
- Make your "REST service" truly RESTful
- Structure REST resources
- Avoid creating WSDL services with unreasonable number of operations
- Be consistent and standard
- Keep the service simple and task-oriented
- Never expose implementation details
- A service should do one thing not many
- Transfer references to data until the data is actually requested
- Do not specify the format as a parameter of a REST service
- Avoid parameter abuse
- Avoid complicated XML input and output data structures
- Avoid anonymous message attachments
- Avoid obscure WSDL
- Test and maintain the service
- Document and register the service
- Standing up services
- Other resources
- Design the service
- Making your Web services compatible with Workflow Management Systems (e.g. Taverna)
There are two main types of Web services: RESTful (REpresentational State Transfer) and WSDL (Web Services Description Language) /SOAP.
RESTful services allow a client to manipulate resources (a resource can be essentially any coherent and meaningful concept that may be addressed) on the server providing the service. WSDL services allow the client to call operations on the server.
Add yourself if you edit this page
Alex Nenadic, Alan Williams, Robert Haines, Anton Güntsch
RESTful services use the HTTP protocol. They are based around the concept of a resource and HTTP methods that are used to access, replace, create or delete those resources. As described on Wikipedia, a common mapping of services is:
Collection URI, such as http://example.com/resources/
List the URIs and perhaps other details of the collection's members.
Replace the entire collection with another collection.
Create a new entry in the collection. The new entry's URL is assigned automatically and is usually returned by the operation.
Delete the entire collection.
Element URI, such as http://example.com/resources/item17
Retrieve a representation of the addressed member of the collection, expressed in an appropriate Internet media type.
Replace the addressed member of the collection, or if it doesn't exist, create it.
Treat the addressed member as a collection in its own right and create a new entry in it.
Delete the addressed member of the collection.
POST is also commonly used as a means of performing a complex get or search operation. The POSTed data specifies the parameters to the search.
Tools to build REST services
The Java API for XML Web Services (JAX-WS) provides full support for building and deploying REST Web services. It is tightly integrated with the Java Architecture for XML Binding (JAXB) for binding XML to Java data and is included in Java 6.
Some guidelines on building REST Web services can be found online.
WSDL/SOAP Web services
WSDL Web services are defined by their WSDL document, an XML format that represents an interface to a Web service. WSDL is machine-readable description of the operations (functions) and parameters offered by the service, i.e. XML message types that the service receives and produces and that get wrapped in SOAP messages exchanged between the client and the service.
WSDL binding describes how a particular Web service is bound to the underlying SOAP messaging protocol (or any other protocol used as a carrier). A WSDL-SOAP binding can be either a Remote Procedure Call (RPC)-style binding or a document-style binding. A SOAP binding can also have an encoded use or a literal use. This gives four style/use models:
There is also a fifth binding style that is commonly referred to as the
document/literal wrapped. Thus, developers have five binding styles to choose from when creating a WSDL file. A good description of the differences between these styles can be found online.
Service creation and deployment guidelines
Design the service
Consider how a service consumer will use the service
“An interface is a contract between data provider and data consumer”, Lincoln Stein, 2002.
Web services are nearly always implemented as an afterthought. Service providers usually already have some local code that they want to expose as a public service. So they generate an interface to it automatically in the "hack and publish" approach. The interface ends up being cumbersome and hard to use and understand, tied to the underlying local implementation and configuration, exposing internal ids, class names and formats.
Service providers should write contract-first services - define the service's interface and data types first the way you want service consumers to see it and then implement the service according to the interface. Also refer to "Contract-First Web Services: 6 Reasons to Start with WSDL and Schema" for more details.
Design your services with compatibility and interoperability in mind
More than 70% of the steps in a workflow are "gluey things" that convert data from one format to the other and less than 30% do some science with the data. If there is a data compliance model in your service domain you should consider coding against it.
There are a few service initiatives to try to define data exchange formats and service ontologies, such as BioXSD Data Exchange Format, EDAM Data and Methods Ontology, BioSharing, VO Table interchange format, etc.
If your service returns data for which well-defined vocabularies exist (e.g. ISO country names, ISO languages), use them and document their use. Client systems will then be able to easily integrate service using the same vocabularies.
Consider reliability and stability of your services
Users value services that are reliable and stable but services decay over time if they are not maintained properly. Reliability and stability increase users' confidence and rating of your services.
Consider how registries and consumers can monitor the service
It is extremely useful for service consumers or service registries to be able to test whether a service is available and whether it is running correctly. It is good practice for service providers to consider how their service can be tested by external bodies. For example, a "ping" operation to check that a WSDL service is "alive". Some registries, such as the BioCatalogue, allow for tests to be included in the description of a service and those tests are automatically run by the registry.
Do not "press a button" on existing code.
It is very easy to think that providing a service can be done by simply running a tool over existing code. This may "tick the box" of making a service available, but the service is likely to be almost unusable by service consumers. When generating a service, it is necessary to think about what the service consumer needs. The best way to do this is to create the services "contract first" i.e. design the service's WSDL or REST and associated document formats, then from those generate the classes and methods for the service; the service methods are then implemented by calls to the existing code. Alternatively (and more commonly) the existing code should be wrapped and annotated so that consumer-oriented services are generated.
Allow the service to be used by consumers who know minimal information about the service
It is easy for the service provider to assume that service consumers will know almost as much about the service as they do. This assumption leads to lack of documentation, strange features (parameters that change meaning or that should not be used together), changes in the semantics of output and input data.
The service provider should assume that the service consumer's knowledge of the service is minimal. The service consumer merely wants to achieve the task exposed via the service.
Create services that do what consumers want to do
Services should expose consumer-oriented tasks i.e. what a service consumer wants to do. Services should not expose provider-implemented operations i.e. what the service provider's code does to perform a task.
If the service that is being exposed corresponds to methods or functions within the service provider's code, then the service consumer is forced to use it in a specific order to achieve a task and to process implementation-specific data.
Make your "REST service" truly RESTful
The concept of Hypermedia as the Engine of Application State and the suggestions described by Roy Fielding are extremely useful when creating a truly RESTful service. "A REST client needs no prior knowledge about how to interact with any particular application or server beyond a generic understanding of hypermedia." i.e. include links to related resources.
Structure REST resources
Avoid creating WSDL services with unreasonable number of operations
There is a number of WSDL services that have a large number (e.g. hundreds) of operations. They make service annotation and usage very difficult. Consider grouping your operations into several services.
Be consistent and standard
Follow service standards for your domain
There are a number of standards for services in particular domains, such as the TAPIR protocol, Web Map Service and the IVOA services . If your service is in a domain that has such standards then it is essential that you follow them.
It should be noted that most of these standards use similar concepts such as the need to deliver a capabilities document that describes the particular data or algorithms that a given service exposes. Such capabilities documents often include the description of the errors that the service can/should return. It is a good idea to deliver a capabilities document even if you are not implementing against a standard.
If there is no current standard for your domain/service, consider creating a group to develop such a standard. It will make the life of service consumers much easier and will ease the adoption of web services in your domain.
|Comment from Anton Güntsch|
For the (existing) recommendation "Follow service standards for your domain" I am not quite sure whether you really want this from a workflow developers point of view. If you take TAPIR as an example (which doesn't have a lot of uptake compared to BioCASE, DiGiR and DwC-A in the community by the way): it might fulfil all the requirements of biodiversity primary data networking but it's hard to understand and therefore not easy to use for persons who come across it for the first time and want to integrate it into their workflows/systems. Do we really want to recommend the use of such "standards"? I would rather say: use the standards of your community put consider adding simplified streamlined access points to make "external" Users happy.
Use a registered format
If possible, use a format registered with the Internet Engineering Task Force (IETF). This is particularly important when the service is sending or receiving "leaf" data. If the format you want to use is not registered with the IETF, then take the small amount of effort needed to register it.
Try to re-use formats
There are a large number of existing formats in which data can be specified. Wherever possible an existing format should be used by the service. The minimal advantage to the service provider of defining their own format is normally outweighed by maintenance costs; there are also inevitable costs for the service consumer in understanding and processing different formats.
Specify the format
If a service is returning XML then the service provider should specify the xsd to which the data conforms. Similarly, if the service takes XML then the xsd of the data should be documented.
If the service is a WSDL service, then the data format should be specified via inclusion and referencing of the xsd within the WSDL. It is not correct to just specify any.
Return data in the format requested by the client
To quote from "Implementing REST Web Services: Best Practices and Guidelines"A resource may have more than one representation. There are four frequently used ways of delivering the correct resource representation to consumers:
- Server-driven negotiation. The service provider determines the right representation from prior knowledge of its clients or uses the information provided in HTTP headers like Accept, Accept-Charset, Accept-Encoding, Accept-Language, and User-Agent. The drawback of this approach is that the server may not have the best knowledge about what a client really wants.
- Client-driven negotiation. A client initiates a request to a server. The server returns a list of available of representations. The client then selects the representation it wants and sends a second request to the server. The drawback is that a client needs to send two requests.
- Proxy-driven negotiation. A client initiates a request to a server through a proxy. The proxy passes the request to the server and obtains a list of representations. The proxy selects one representation according to preferences set by the client and returns the representation back to the client.
URI-specified representation. A client specifies the representation it wants in the URI query string.
Note that for URI-specified representation, the representation is specified as an extension to the URI, not as a parameter of it.
When the service provider receives a request for a resource in a specific format (character set etc.), it should either:
- return data in that format, or
- return a HTTP 406 Not Acceptable error
Points 5 and 6 are not at the same level as 1-4
It should never return data in a default format that does not match that specifically requested. Incorrectly returning data in a default format prevents the client from being notified of the format problem and also causes the client to process data in the wrong format.
REST GET services are very similar to fetching pages for a web browser. Some service providers simply deliver an HTML document corresponding to what they would return when the corresponding web page is fetched.
Unless the service is requesting a HTML document, the service provider should not return an HTML document. The service should return the data that underlies the HTML document.
Because it is possible to use web-content providers as if they are REST services, it is possible that apparent REST services within a workflow will actually be fetching HTML documents. Wherever possible it should be made clear that it (what is 'it') is not a "proper" REST service.
Be consistent with what is returned
Some REST services return JSON for certain resources, a choice of JSON or XML for other resources, and XML for other resources. This is very confusing for service consumers. Services should be consistent in the formats in which they can return data.
When a service can return a resource in alternate formats, the data returned must (as far as the formats allow) contain the same information. For example, if the JSON that is returned contains the name, address and telephone number of a person, so should the XML.
Return the correct error codes
Service providers should read the HTTP Status Code definitions and familiarize themselves with them. (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html)
Services should only return error codes as specified in the HTTP specification. Creating error codes that are specific to a service breaks the HTTP specification and prevents clients from using the service.
Services themselves should never intentionally return any error code in the 500 family - these are reserved for use by the harness within which the service is running (eg Apache). This way you know that if you see a 500 error code then it is not your service that is broken but something further down the stack.
Service providers should not only document what errors are returned, but also check (as far as feasible) what happens when unexpected requests are received.
When authorization is required, return a 401 (Unauthorized) error, not a 403 (Forbidden). In English the Forbidden seems appropriate but it is overloaded. In the HTTP specification, a client gaining authorization will not resolve a Forbidden error.
It is not uncommon for HTTP 500 code (Internal server error) to be returned when attempting to access data that does not exist. The correct code should be returned, for example 301 if the resource has been permanently moved.
Use standard security mechanisms
There are standard mechanisms for limiting access for both WSDL and REST services, for example WS-Security and HTTPS. It is almost never a good idea for a service provider to use a novel security mechanism.
For example, some services require that a token is passed as one of the input parameters. However, that token is commonly constant for a given service consumer. In order to call the service, it is usual for service consumers to hard-code the token value. When the service is included in, for example, a workflow, the value of the token is shared along with the workflow. If a standard mechanism such as HTTP username+password had been used, then the security credentials would remain hidden.
Comply with Web Services Interoperability Standard (WS-I)
The WS-I link is not the most informative page. Perhaps one of the other pages might be more useful:
"WS-I profiles define how existing WS (Web Services) specifications should be used in order to achieve maximum interoperability. The profiles effectively clarify the way existing standards should be used because the final documents were not clear in places or the flexibility they allowed was leading to interoperability nightmares. If the rules of WS-I profiles are followed, it is expected (but not guaranteed) that the resulting deployments would interoperate (at least as far as the underlying infrastructure is concerned)."
You can choose to make your Web service non-compliant, depending on your needs. For example, encoded style (RPC/encoded), SOAP over JMS protocols, and secured Web services, do not comply with the WS-I Basic Profile.
Keep the service simple and task-oriented
Never expose implementation details
The internal details of the service implementation should be hidden, as far as possible. For example, just returning a serialization of a Java object is not a sensible or usable response to a service call.
The exposure of even internal ids can cause problems as the services may assume that an internal id is passed in. How is the service consumer meant to know the id?
Wherever possible, implementation classes, data and ids should not be exposed by the service. The service provider should look at what is actually returned by some example service calls and consider if they are actually usable.
A service should do one thing not many
Polymorphism is when the task performed by the service is determined by one of the parameters to the service call. For a polymorphic service, the validity, meaning or permitted values for other parameters may depend upon the value of the controlling parameter. It is possible to have polymorphic WSDL or REST services.
For example, consider a polymorphic service that allows you to query one of several databases, books or films, then a query may appear as:http://www.example.com?database=film&director=lucas
where it is not valid to specify the director for a book and the semantics of author switches from writer to screenwriter depending upon the database.
It is far better to have separate resources corresponding to, in this case, the separate databases with the correct parameters specified for the resources.
Transfer references to data until the data is actually requested
It is normally better to transfer references to data, for example a URL, rather than the actual data, especially when the data is of significant size. The only exception to this is when the data is explicitly fetched, for example by a HTTP GET operation.
Do not specify the format as a parameter of a REST service
The HTTP protocol used by REST has a mechanism for specifying in the message header the format that the data should be returned in. For example:Accept: application/xml
REST services should not specify the format as part of the URL or message body e.g.http://www.example.com/fred?format=xml
This is not RESTful. Such a resource should be requested from http://www.example.com/fred with Accept header application/xml.
The use of a format parameter can cause problems. For example, if the service consumer requests "http://www.example.com/fred?format=xml" with "Accept: application/json" then it should be impossible for the transaction to take place.
An alternative is to use URI-specified resource representation. This would use the extension .xml in the URI, for example http://www.example.com/fred.xml
Avoid parameter abuse
Many services abuse their input parameters to pass the information to the service that changes, for example, the method invoked, the database searched or the content type of the returned data. This parameter abuse hides the resources used and is particularly present in REST services (also see Avoid polymorphism and Do not specify the format as a parameter of a REST service points above).
For example, not all services follow the content negotiation principle whereby you can ask for the resource in different formats just by changing the HTTP header value to another known MIME type; some services expose the content type as an actual parameter to the REST call.
Example of using a service as a parameter:http://www.example.com/invoke?service=x&method=y
Avoid complicated XML input and output data structures
XML data structures with a high level of nesting are hard to read and understand. Try to keep your data structures clean and simple. Do not expose implementation details in your resulting XML resulting from automatically generating services from existing code.
An example of a not very good XML is shown in the image below.
Avoid anonymous message attachments
(for WSDL services)
Avoid obscure WSDL
Even though WSDL documents are not meant for human consumption, avoid WSDL documents such as <wsdl:part name="in0" type="xsd:string"/>. Parameter name "in0" means nothing to a user trying to invoke your service.
Some of the sub sections above get right down to the nitty gritty of the service styel (WSDL/REST) while some other bits are very generic.
Test and maintain the service
Assess and document performance limitations
Predicting performance bottlenecks for service can be difficult. There are however tools such as Apache JMeter (http://jmeter.apache.org/) which can be used to assess the behaviour of services in a structured way. Detected performance limitations should then be included in the service documentation.
Use your services in your own systems and workflows
Many service implementations lack a clear use case and are just exposed in addition to a human-readable portals for an unknown future application. To ensure that service provide the expected functionality and performance developers should try to use them as much as possible in their own systems such as web-portals using these services rather than local APIs.
Document and register the service
Describe the service
Many services are poorly described. Even when the services are registered with, for example, the BioCatalogue, only minimal effort is put into documenting them. An undocumented service is an unusable service. The documentation should include, at least,
- description of the task that the service performs - what it does from a service consumer's point of view
- what can be used as input, with example values and description of what happens if the value is not specified
- what will be returned as output, including example values
- possible error messages, including what they mean from the point of view of the input data and the intended task. The error message should not be described by relation to what has gone wrong in the provider's code
It is very useful if the documentation includes:
- example programs, scripts and/or workflows (including example input data and results) that use the service
- other services that work well with the service
Document any choreography
Some services require that they are called as part of a group and in a specific order - this is often termed a "choreography". An example of services requiring a choreography are the WSDL operations generated by Soaplab. If your services are expected to work as part of a choreography, or in a more extreme case, will not work except in a choreography, then you must document the choreography. The documentation should include the description of the order in which the individual services must be called and also, as far as possible, what will happen if services are called "out of order".
Register your service
Unless you intend to keep your service private, it is very good practice to register it. There are a number of service registries. Some of them such as seekdaare general and can be used for any WSDL service. Other registries are domain-specific, for example the BioCatalogue for Life Science Web Services or one of the registries of the International Virtual Observatory Alliance (IVOA) for archives of Astronomical data.
Consider separating availability information
If your service is currently unavailable, then normally service consumers will have no idea what has gone wrong or when the service will re-appear. Some services, such as those registered with IVOA, allow the registration of a URL where availability information is held, with the actual services preferably being at a separate address.
If the registries where you register your service do not support availability information, then it is a good idea to press them to support such information. Although there is currently no standard mechanism for describing availability information, one such mechanism is ...
Standing up services
Consider cloud-based service provisioning
Hosting your services on a cloud provides greater reliability, stability and capacity for users in comparison to services run on a desktop PC under someone's desk.
- Implementing REST Web Services: Best Practices and Guidelines
- Common REST mistakes
- Contract-First Web Services: 6 Reasons to Start with WSDL and Schema
- A book by Glenn Hostetler, Sandor Hasznos and Christine Heron: Web Service and SOA Technologies - Protect your project and carrier by understanding common mistakes
Making your Web services compatible with Workflow Management Systems (e.g. Taverna)
In addition to the general advice on building Web services above, there are still a few things to bear in mind when building your services to work workflow management systems such as Taverna. Please refer to the "Web services in Taverna" document for details.