coarsegrained

web services course redux

Teaching the web services class again this fall.  The revised syllabus is at http://rosetta.sims.berkeley.edu:8085/sylvia/f05/view/290-3.complete

This semester will focus more on mobile web services, including exposing some of the mobile web services technology we're building at Good Technology.

Versioning Web Service Information in UDDI

This blog has been a great source of feedback from customers and industry colleagues. I'd like to use that channel a little more proactively now.

We're working with the UDDI TC on approaches to versioning (as the public OASIS UDDI meeting minutes show). We've currently identified that we want to take any web service object in UDDI (business services, binding templates, and portType and binding tModels generated following the "Using WSDL in UDDI" Tech Note) and put versioning information on any of them. Our current thinking is that this includes version numbering (usually a major.minor number but any content should be alright), revision or build information (of arbitrary content), milestone information (such as "latest revision"), and "component information" that can be used to logically tie together multiple objects. Unchecked taxonomy tModels can be created for each of these types of information. From any UDDI web service object (e.g. a portType tModel representing an interface definition) categories (keyedReferences) that refer to those tModels and whose keyValues reflects the version, revision, milestone and component.

All this is straightforward enough. A few questions:
- is it acceptable that this won't work for V2 bindingTemplates (categories can only be placed on UDDI V3 bindingTemplates)? Most versioning may take place on the portType tModels (the interfaces) so maybe this is okay.
- should the version be arbitrary content (any string) or should it be allowed to be numeric (possibly with a majorVersion category and minorVersion category)? Since UDDI will get range searching soon enough, this might be attractive. It is however, more complex (more categories necessary) and may not match with the version labeling methods of some shops.
- are there other semantically meaningful milestones that should be "built-in" besides "latest revision", "latest version".

I'm very curious how companies have versioned their web service information in their registries (UDDI-based or otherwise, from Systinet or elsewhere).

WS-Eventing feedback workshop

Excited that its going on tomorrow at Tibco Palo Alto and that I have an excuse to be home in the Bay area yet a few more days. WS-Eventing is a great spec. I've provided some feedback on this blog on it already. I have more substantive suggestions for tomorrow.

Below is a sample C# command line utility I wrote tonight that allows you to subscribe a URL-specified endpoint to XPath-specified document content. It does this by sending WS-Eventing Subscribe messages to a WS-Eventing supporting Content Based Routing service. Full source (with comments even!), WS-E WSDL file, VS project files and executable is also available here. Works against our WS-Eventing implementation nicely it seems.

* Shouldn't be necessary to have such caveats on a personal blog, but it has not been tested very much at all, and for all I know has all kinds of bugs (certainly should be doing more sanity checking of the SubscribeResponse than I do below). The only real use of it will be that some variation of it will probably end up as a sample in an article or tutorial somewhere. In other words, its true CTO-quality code...

Haven't tried it against Casey Chesnut's WS-Eventing implementation (which requires Microsoft SQL Server) since I don't have SQL Server on any systems here (shocking since I was on that team at Microsoft). Any other implementations to interop test this against? I'm sure I'll find out a few tomorrow...

Couldn't resist showing how simple the program gets to be due to the straightforwardness of WS-Eventing and how easy it is to subscribe to document traffic. Now, getting source code to appear nicely on a blog is another story, as the indentation is totally lost below :-(.

using System;
namespace CBRSubscribe
{
class MainClass
{
[STAThread]
///////////////////////////
static void Main(string[] args)
{
if (args.Length<3)
{
Console.WriteLine(
"Usage: CBRSubscribe <web service for notifications> <filter expression> <CBR URL>");
return;
}
bool result;
result=Subscribe(args[0],args[1],args[2]);
if (result)
Console.WriteLine(
"Endpoint {0} is successfully subscribed to documents matching {1} with CBR service {2}!",
args[0],args[1],args[2]);
else
Console.WriteLine("Subscription failed!");
return;
}


static bool Subscribe(string notifyAddress,string subscriptionFilter,string cbrServiceURL)
{
CBRSubscribe.CBRService.Subscribe subscription=new CBRSubscribe.CBRService.Subscribe();
subscription.NotifyTo=new CBRSubscribe.CBRService.EndpointReferenceType();
subscription.NotifyTo.Address=new CBRSubscribe.CBRService.AttributedURI();
subscription.NotifyTo.Address.Value=notifyAddress;
subscription.Filter=new CBRSubscribe.CBRService.MessagePredicateAssertion();
subscription.Filter.Value=subscriptionFilter;

CBRSubscribe.CBRService.Eventing objEventing=new CBRSubscribe.CBRService.Eventing (cbrServiceURL);
CBRSubscribe.CBRService.SubscribeResponse subscribeResponse=
new CBRSubscribe.CBRService.SubscribeResponse();
subscribeResponse=objEventing.SubscribeOp(subscription);

// sanity check the subscription results
bool result=true;
if (!subscribeResponse.Id.StartsWith("uuid:"))
result=false;
return result;
}
}
}


UDDI Face to Face Meeting

Last week's UDDI Face to Face meeting in San Francisco (hosted by Maud Cahuzac and Shishir Garg of France Telecom) was extremely productive. All of the details and minutes are public up on the OASIS UDDI site (there's nothing here that you won't see there in other words). But to summarize:
- Concrete progress was made on nailing down a method of range searching for category values. This is particularly crucial for some proposals we are working with others on for storing management information associated with web services in the registry.
- A simple method of performing boolean searching with category bags and combinations of them, without requiring a fullfledged new query language, looks like it will be introduced. The default of reducing everything to conjunctive normal form by default (implied OR for values within category bags, and implied AND between category bags) is great way to get arbitrarily complex queries without forcing learning and implementation of new "keywords"
- There was a proposed simplification of requirements of taxonomies to concentrate on identity between nodes (within and across trees) and parent/child relationships. It needs to be verified further but I think its a step in the right direction. After the goals are agreed upon, I would imagine it should be quick work to come up with a representation: whether its based on an existing format such as a subset of OWL or RDF, or a new format entirely.
- Methods of representing ACLs were worked on, which is an important thing to happen I think, though I wasn't attending that track of the session.
- It was agreed that it would be value for the UDDI TC to put out a Tech Note, in consultation with WSDM, for storing quality of service information in UDDI. Fred Carter of AmberPoint and I will be working on this, with hopefully lots of guidance and help from other management vendors on WSDM and sage advice from UDDI TC members.

Speaking of sage advice, Tom Bellwood of IBM did a great job driving topics to quick, practical and shorter term closure. All of this bodes quite well for UDDI in general. In my days as VPE at Commerce One I remember not being enthusiastic about UDDI V1. But UDDI V3 is a very solid specification as it stands. And the enhancements discussed seem like they should be able to be layered on top of V3 without major changes and thus are perhaps not too far in the future in being available to customers. Whether its Systinet or IBM or Microsoft's UDDI registry, companies should strongly consider using a current UDDI implementation as the central site for storing all of their web service information.

WS-Eventing spec released

I was very excited to see the WS-Eventing specification released this week. We've been looking for something just like this for a number of our purposes. As one example, the body of subscription request messages has just the things we need to specifying a content routing subscription:
- where to send content notifications (the content itself)
- where to send notifications of subscriptions ending
- when subscriptions should expire
- a method of specifying filters on the subscriptions

It also has, in the body at least, no baggage that we don't need. By contrast, we have seen other content routing systems require subscribing users to set up rules for the content routing first, and then allow users to register interest in those rules. This creates a manageability issue for subscribing users of having to either identify and search among existing routing rules, or follow a two step process of creating rules and then establishing subscriptions to them, an unnecessarily complex process. I'll be posting more on attributes of easy to use content-based routing later, and what content-based routing adds to the pure web services world that it doesn't have now. Suffice it to say in the meantime, that WS-Eventing has what is needed to do content-based routing subscription and notification.

This is very good news. But I also see a few flaws in the current WS-Eventing specification. WS-Eventing uses the WS-Addressing spec intelligently for several things:
- the endpoint references for content and subscription end notifications are of WS-Addressing EndpointReferenceType
- this includes use of WSA ReferenceProperties within those endpoint references that allows followon messages (subscription responses, content notification responses) must include in their message headers (literally without any surround WS-Addressing or WS-Eventing related header)
- to relate subscription requests and responses (and other request/response message pairs). Specifically responses will have a WS-Addressing RelatesTo header that matches the MessageID content of the request.

These uses make sense, and its great that we don't have to learn a one-off method for these capabilities (endpoint references, reference properties needed by requestors, and correlation of messages). But there remain some unexplained and sometimes inconsistent issues around WS-Eventing's use of WS-Addressing headers:
- Is use of the WS-Addressing headers actually required? It is implied by various aspects of the spec but not actually spelled out. There could be quite a bit of utility for many people to allowing these WS-Eventing conversations to take place via synchronous generic SOAP http requests, using only the SOAP Body content specified (though that content of course correctly uses types from the WS-Addressing spec).
- If use of wsa: headers is required, there is more information that needs to be nailed down in the spec. The spec currently says that wsa:RelatesTo in the header of SubscribeResponse, must be value of the wsa:MessageID of the corresponding request. It doesn't say anything about the wsa:To. Presumably (if use of wsa: headers is required) it must be the value of the wsa:ReplyTo of the request. If the SubscribeResponse is coming back in the http response from a synchronous SOAP request, then perhaps the wsa:To is not necessary. If it is necessary (and population of all wsa: headers is required), then the correct population of it should be described, if only for the sake of consistency.
- There is a dangling element definition called SubscribeTo that is not used anywhere in the schema. I suspect that this is because some method of identifying who to send subscription responses when not using WS-Addressing headers was originally planned. It would be necessary when you didn't have wsa: headers if you were asynchronously responding to a subscription request.

Overall, I think there is some value to allowing the exchange of these WS-Eventing messages (subscribe request/response, unsubscribe request/response, etc.) over generic SOAP non-WSA-supporting transports. This could be done by requiring a synchronous http exchange to effect them without having to do explicit correlation of RelatesTo/Message and To/ReplyTo elements. And yet we could still use the relevant WS-Addressing data types where they help (as noted above). Nevertheless, I can also live with required use of WS-Addressing headers or even a fullfledged WS-Addressing-capable transport (we'll be building a very early an implementation of such anyway). I just think that the WS-Eventing specification should be a little clearer in that case.

These reservations aside, WS-Eventing is a very exciting development. Beyond the fact that it is very good specification (I think the WS-* specs are actually getting better over time), by facilitating the creation of content routing services, it introduces for the first time to the web services world the capability of sending messages for which you do not know the recipient. This is a key building block to enabling rapid integration of many web services and applications around individual business events.

A Rose By Any Other Name

I have gotten some good feedback from various quarters on the term "web services bus". This is definitely not the only possible name. We do believe that it conveys both the orientation to pure web services and, with the word "bus", the distinction between the current generation of point to point web services integrations.

But there are other possible names. For example the following names have been proposed by various other web services analysts and pundits:

• services infrastructure layer- a very good report from the New Rowley Group. Very close to the web services bus in concept. The characteristics are described as: decentralized, deployed incrementally, built on open standards, and enhanced by a plug-in market: all of which sounds very familiar. Its especially gratifying to see the emphasis on plugins. It does not talk about message delivery capabilities or the move away from point to point style web services integration.
  
• service grids- as described in John Hagel's and John Seely Brown's paper Service Grids: The Missing Layer in Web Services. It is less specific about the role pluggable services can play. It also has an unfortunate overlap with grid computing: a separate topic entirely.
  
• pure web services ESB - I have seen this term proposed as well. It doesn't quite roll of the tongue though. It also doesn't convey all of the differences of what we've been proposing from other ESBs. I think it would be better to pick more of a "greenfield term" that doesn't already have a lot of associations with it (perhaps also an argument against using the word "bus" at all in the term).
  
• fabric, services fabric, web services fabric - I've seen the word fabric now used to describe enough different offerings (from WebMethods/TME, Sonic, even Microsoft) that I'm not sure what it means. The best distinction from bus that I've seen is that a fabric is highly distributable with no centralized nodes required. In essence, it seems to be a description of an architecture for supplying web services-related services, such as failover. Any such infrastructure will need at least a logically centralized registry. However, if there is consensus that this word is better for describing the attributes we have been discussing below, then we should adopt it.
  

I think the "adjective prefix" can either be "services", "web services", "services integration", or "services infrastructure". I don't have a strong preference except that "web services" may convey the orientation toward open standards.

For the "noun", the options are (at least): layer, grid, fabric, and bus. Bus conveys the move away from point to point, while having some baggage from the old EAI days. Grid has perhaps too much association with grid computing. Layer is perfectly accurate but doesn't really give too many clues as to what the layer does. Fabric may or may not be the term to use depending on what meaning it has evolved to.

So there you have it. Pick one from column A and one from column B: "services integration bus", "web services layer", "services infrastructure grid". I don't have a strong preference either way. I'd love to hear other nominations.

The Web Services Bus

I've discussed in earlier posts why its important to facilitate building loosely coupled, coarse-grained document-oriented web services. I've more to say on that subject as time goes on. But I've also gotten feedback in various forms from customers, partners and attendees of talks I've done to the following effect:

Thats great, I already agree that this is the best way to build web services. We're already doing that: defining common shared libraries common XML document schemas that the whole enterprise uses. But once we have these shared vocabularies does it really make sense to keep building out peer to peer SOAP connections? If we have shared document models, can't we use that to enable some form of hub-based communication model?

Of course the way people say this tends to vary. Sometimes the feedback is "hmmm, shared object models. Sounds like the old days of EAI". In effect the answer to all of these statements and questions is "yes". And the Systinet toolset can be used in these scenarios. We have JMS support so you can "build your own web services hub" with Systinet on top of JMS. The question really is, is there more that can be done? Can we use the tools and APIs available for web services protocols to work on top of common messaging buses (JMS, proprietary buses, and, with the advent of WS-ReliableMessaging, SOAP over http) to provide a hub for web services communication.

The MOM market has a number of common paradigms that it supports in synchronous and asynchronous messaging, for two party and multiparty scenarios. For two party asynchronous communication, some of the scenarios include:

• Asynchronous request-response. The bus infrastructure can create correlation variables to help manage this scenario.
• One way message to a web service from the client
• Solicit response – services send requests to clients. The bus creates correlation variable and client sends back response
• Notification – web service sends a message to the client

For multiparty asynchronous communications, a messaging bus provides great value in assisting with simultaneous communication with multiple other endpoints. The two primary methods of performing this are:
• Multicast - which sends identical messages and content to multiple named parties where each parties status and response is tracked individually. Value-added services such as content transformation or routing can plug into multicast events to optimize the communication to individual parties. The necessary correlation IDs are generated to be able to track each participants status and responses
• Broadcast – messages are sent to some form of aggregated list such as a topic or a group. Responses are not tracked.

For long running transactions (for two or more parties), the bus can supply support for and management of long-running web services conversations. It can generate and manage the conversation IDs that are necessary for tracking these conversations. Many endpoints may not have the capability of generating conversation IDs. The bus can provide calls to generate the conversation IDs transparently and to allow endpoints to check on the status of the conversation at any time. It can support several methods of communicating the conversation meta-information including WS-SecureConversation.

Finally, even though the bus is primarily built to support loosely coupled integration, it can still add value to the world of simple synchronous request/response. And such support may be necessary when sync communication is used in a longer conversation being managed by the bus. That is, if synchronous communication is all that is desired for some particular integration a MOM may not even be used. But often a long running conversation may include one or two synchronous web services invocations to facilitate the process. The bus can support this scenario and even provide value-add in providing error handling and failover for such scenarios that a simple synchronous web service client might not have the time or capability of providing. Even more interesting: the bus (or a service on the bus) can provide the capability for endpoints that invoke simple synchronous services to treat those as if they were asynchronous. Endpoints can fire and forget and communications services offered by the bus will call them back when the synchronous execution is complete and provide the data in the response.

The question really is: which of these are important in the web services world? Even if you build a web services layer on top of very full-featured MOMs, do you need all of the possible methods. I'm very interested in feedback from people doing web services-based integration (and even application integration using older technologies) about which of these capabilities continue to be important.

Web Services SIG

Enjoyed giving a talk at the SDForum Web Services SIG last night. Got lots of good comments at the end about XML document-oriented development. Plan to move up some discussions of what might be required in development tools for this area to happen sooner.

Indigo at the PDC

Interesting developments on Indigo at the PDC. Its been a while since I've missed a PDC for any reason, but this one seems particularly bad one to miss due to the unveiling of Indigo. Sort of makes me miss the days of speaking there. Drew Marsh is doing an excellent job of realtime blogging this. Of particular interest are the posts on Indigo building service fundamentals and secure, reliable, transacted services. But the session I most enjoyed reading about was the Indigo opener. Indigo, as most of you probably know, is Microsoft's technology to make it easier to design and implement web services. Although it will ship with Longhorn (the next MS OS), it does not require it. The WSE frameworks that have been posting for a while to support various WS-* SOAP enhancements will get folded into Longhorn deliverables. In the meantime the WSE is what to work with.

Its great to finally be able to talk about this stuff. I talked a lot with MS in my Commerce One days (and even my Microsoft days;) about what they were doing, but could never explicitly reference it.

Of particular importance from reading various blog notes are some themes I've been talking about for a while and also of importance to Systinet:

• share schema and contract not class - don't worry about the classes someone else is implementing their applications and systems with. Agree on the schema. Write it beforehand. Then embed references to it in your WSDL operation's input and out messages.


• loose coupling is important - synchronously executing someone's SOAP RPC-exposed method is not the right way to do things


• respect system, process, and organizational boundaries - transparent remoting is dead

 

There's also a recent MSDN article from Don with similar content.