The trouble with SOAP

Ripples continue in the debate about whether SOAP is an over-engineered, unnecessary addition to the web services infrastructure. This week, Jeremy Zawodny (who in his dayjob is one of the webheads behind finance.yahoo.com) posted a weblog entry referencing recent articles about REST, the URI-based alternative promoted by anti-SOAP activists:

"I've always wondered about the various protocols for so-called 'Web Services', namely SOAP and XML-RPC. What bothers me is that they're called Web Services in the first place. It feels like there's no Web in them at all. The Web has never been about APIs."

He goes on to cite the Paul Prescod article that I mentioned earlier this week (plus an earlier one), and concludes: "Having been part of the Internet in one way or another for the last ten years or so, I find that his view of what could be just feels like the right solution — for now."

All this despite having read Jon Udell's InfoWorld article from May, Hyperlinks matter, in which Jon described the debate, but also reported that a rapprochement was likely — since SOAP toolkit developers now plan to add the capability to make services available as URIs, thus removing one of the main objections made by REST advocates.

Why then do people still feel driven to take sides? Perhaps because, as Jon concludes, "RESTians see through the lens of hypermedia, while SOAPistas wear client/server spectacles." A final link in Jeremy's piece is to diveintomark's helpful anthology of the original debate back in April, which comes down firmly on the client-server side of the fence, based on the irresistible argument that it will be enterprise demand that drives adoption of web services.

I'm sure that's true, but it doesn't necessarily follow that something is right just because a lot of big companies start doing it. The important thing to remember is that the primary role of SOAP is to expose existing application functionality as a web service, and so enterprises will adopt SOAP because it is a low-cost, non-disruptive way to migrate their existing application infrastructure into the web services environment — as Jon described last week in another article, Legacy assets meet SOAP: "Virtually every software asset can now be offered, or soon will be able to be offered, as a web service."

Fine, but that's just a transitional step — quoted in Jon's article, Cape Clear founder Annrai O'Toole notes that wrapping legacy applications in SOAP "is effectively screen-scraping." It still doesn't fix the core problem, which Doug Kaye alluded to in his weblog this week (a propos of a very useful diagram he's drawn, representing "the layers in the web-services pyramid"). Here's what he said:

"The top of the pyramid is the world of business semantics: industry-specific protocols, formats and documents. This will be the final frontier for standardization, and even then it will be dominated by specialty vendors and not-for-profits that operate vertical hubs providing centralized workflow automation and directory services."

It stood out for me because it reminded me precisely of the passage that had struck me most in Paul Prescod's REST article:

"The technology you use to move bits from place to place is not important. The business-specific document and process modeling is ... What we need are electronic business standards, not more RPC plumbing. Expect the relevant standards not to come out of mammoth software vendors, but out of industrial consortia staffed by people who understand your industry and your business problems."

This suggests a timeline over the next several years that runs something like this:

1. SOAP drives the mainstream adoption of web services to reach critical mass


2. A lack of standardization of business process semantics and ontologies both within and across industries becomes the next barrier to overcome in automating processes between enterprises.


3. Finally achieving a standardized model for referencing business processes paves the way to adoption of a streamlined, URI-based architecture for web services.

If this timeline comes to pass, then both sides of this debate will have been proved right, although at different stages. SOAP is what we need now, but I have a hunch (and it is just a hunch) that where we'll eventually end up will probably look a lot more like REST.
posted by Phil Wainewright 1:32 PM (GMT) | comments | link

You mean, you couldn't do that already?

One of the disconnects that emerges when talking about web services is that developers seem to be surprised and gratified by being able to do things that don't seem that big a deal to non-developers. Here's BEA's director of engineering George Snelling, interviewed on SearchWebServices, giving an example of how the newly-released Web Services Workshop (formerly codenamed Cajun) has helped BEA internally:

Internally at BEA, we have a bug-tracking system for building our own products, and we've been exposing the APIs for that bug-tracking system as Web services. As soon as we did that, we were able to do all sorts of fancy things, such as include the list of bugs that a particular developer has in a window in his or her IDE. So we can pull information directly out of our bug tracking system and display that as an integrated window called My Current Bugs, without interfering with the underlying business logic of that core application. We could probably make it possible for our customers to enter service requests directly into the bug-tracking system as well.

Well, excuse me, but shouldn't the bug tracking system have been doing that already? You mean you designed it without giving developers a view of their current bug status? So what was it doing before these web services came along? And exactly how much had we spent on developing this amazing system that collected information no-one actually used?

The disconnect here is that business managers want automated systems that simply get the job done more efficiently, and they've constantly been surprised and disappointed by the extent to which technology has failed to achieve that objective. Whereas technologist are constantly surprised and pleased when they get something — anything — to work for the first time, and they've always been pissed when the response from business managers has been, "So what?"

I'm afraid it's the technologists that are going to have to amend their ways. They've become so used to winning small triumphs against the odds that they haven't noticed things are starting to change. Computing technology is maturing to the point where the odds are starting to be stacked in favour of success, and big triumphs will begin to be routine. That means that, instead of earning huge rewards for barely meeting expectations, developers will find that exceeding them day by day will become simply part of the job.
posted by Phil Wainewright 2:08 AM (GMT) | comments | link

Security is not a technology barrier

Security is a word that has many meanings. When customers cite it as an objection to web-based technologies, the meaning they have in mind often has more to do with the comfortable reassurance of snuggling up inside a warm blanket than it has with notions like strong encryption and digital certification.

Doug Kaye's weblog last week cited a useful clutch of eWeek articles about digital identity and single sign-on. It's interesting that both Jon Udell and Brent Sleeper have since chimed in to say that, although the conventional wisdom is that security is a big barrier, in fact it probably won't be in the way that people expect. As Doug summed up yesterday, "We know how to solve the security problems. They're no longer rocket science."

So when someone says that they're not adopting web services because of security concerns, what they really mean is, they're just not comfortable with the idea of web services yet. Security is a convenient objection they cite in order to deflect attention from the real reason. Whereas a company that sees the benefits to be gained from deploying web services will just go ahead and do it in a controlled environment, where they can easily fill in the security gaps in today's web services architectures.

You know, I get quite steamed up about this. I nearly bit someone's head off at a conference earlier this year for interrupting my lunch to make the lame observation, "But security's a big problem with web services, isn't it?" No, no, no, no, no. Security is only a problem if you obsess about it. Get a life.

Having said that, digital identity is important, and seeing that Digital ID World has started publishing an RSS feed, I'm pleased to say we have today added it to the Loosely Coupled news aggregator page.
posted by Phil Wainewright 4:48 AM (GMT) | comments | link

The true meaning of loosely coupled

Loosely coupled is a mindset, not a machine state. It's about how you manage your business:

"Management, not technology, is the key to unlocking the value in processes. Technology can improve communications among business partners but doesn't fundamentally change how they manage those processes."

The quote comes from Loosening up: How process networks unlock the power of specialization — a marvellous article in The McKinsey Quarterly by the illustrious trio of John Seely Brown, Scott Durchslag and John Hagel III. It outlines the merits of swapping tightly coupled management systems for loosely coupled ones when engaging in collaborative business relationships:

"Companies at the cutting edge of process management handle critical cross-company processes as though they were networks rather than production lines [my emphasis]. For core operating processes such as the management of supply chains and customer relationships and the development and commercialization of products, these cutting-edge companies have swapped their tightly coupled processes for loosely coupled ones, thereby gaining much-needed flexibility and improving their performance in the bargain"

Unfortunately, conventional wisdom in business and IT is skewed in favor of retaining a tightly-coupled mindset and management style. Bear this in mind when reading about and discussing business process management and web services, because that skew is often so ingrained that it's hard to resist its influence.

Take for example this otherwise excellent article from Web Services Journal on Why Orchestrating Business Processes is Key to Web Services. (BTW, my thanks to orchestration vendor Collaxa, whose weblog is always a mine of useful resources, and which pointed me to both these articles). The article uses the analogy of a performance of a symphony to explain orchestration, thus falling into the trap of perpetrating a tightly coupled view of business process management, in which every movement is known and mapped out in advance. It's a good overview of the topic, but fails to take account of the unscheduled improvisation by individual players that is a characteristic of successful business process networking.
posted by Phil Wainewright 5:32 AM (GMT) | comments | link

Web services the wrong way

Expunge the term API from your web services vocabulary. The very concept of an application programming interface is anathema to a successful web services deployment. That's why we have WSDL and UDDI, which expose functions, not interfaces.

So while I agree with Richard Perkett, CTO and co-founder of online salesforce automation provider Salesnet, when he tells ASPnews that "Tomorrow's applications will be constructed using multiple Web Services from various best-of-breed applications like Salesnet," I find myself disappointed with his description of how it will happen.

Salesnet has just announced a new version featuring a so-called 'web services API', which, according to Perkett, "will allow developers to integrate data programmatically." As far as I'm concerned, that's not the right language to use when talking about web services. The strength of the web services model lies in loosely coupling processes together, and any attempt at programmatic integration quite simply cripples that capability.

But maybe I'm being unfair to single out Salesnet for criticism here, and perhaps the fault lies with Microsoft and IBM for promoting web services as a means of preserving existing applications rather than ultimately replacing them. A highly technical yet thought-provoking article by Paul Prescod for XML.com in February this year made a strong case for developing web services around REST (Representational State Transfer) rather than SOAP. "The technology you use to move bits from place to place is not important. The business-specific document and process modeling is ... what we need are electronic business standards, not more RPC plumbing."

The key point of the REST approach comes out even more clearly in his response later on to a reader's comment:

"Any real business transaction must span many messages sent back and forth. These messages must refer and relate to each other somehow. SOAP and WSDL require you to build this relation yourself in a proprietary way. REST uses the standardized Web-way, which is the URI-bearing hyperlink. This has nothing to do with user interfaces and everything to do with building relationships between information components ... WSDL throws away all of the 'loose coupling' advantages of XML."

posted by Phil Wainewright 4:28 AM (GMT) | comments | link

The Web is the computer

Finally the message is getting through — the Web is becoming a single, shared infrastructure for collaborative computing, and that changes the nature of software for ever. Sun VP and general manager for the Solaris operating system, Anil Gadre, has written a column for CNet explaining why computer operating systems are irrelevant:

As this shift in software continues from shrink-wrapped product to online services, the rules of the game are changing. It used to be that developers tried to sell millions of copies of a program, each installed on a single machine with a single user. Now the challenge is to build a single instance of an application to serve millions of users over the network.

The aim of the service-delivery platform is to make it just as easy for developers to create a large-scale service as it was to create a single shrink-wrapped program for a standalone PC. Developers need to know that certain components are always going to be there — a directory, a network file service, an application server ... these components may also come from a variety of companies offering open-standards-based technology, so long as they present a set of core services that developers can count on.

I'd quote the entire article if I had the rights. Go read it.
posted by Phil Wainewright 11:24 AM (GMT) | comments | link