Most web services projects are missing out on choreography, a vital extra layer that assembles multiple services into coherent business processes.
| ||express delivery|
| print comment|
|Choreography directs the connections between separate application resources to create productive business processes:|
- Also known as orchestration, it manages links between separate activities
- Enterprises use it to extend and combine the functionality of existing IT assets
- Standards such as BPEL are bringing it into the mainstream
- It provides a business-level view of automated processes
- Adoption can start anywhere there's a gap in process automation
Glossary terms: choreography, orchestration, BPM, BPEL4WS, BTP, lookup tool
But some enterprises are beginning to wake up to its potential. When Loosely Coupled interviewed London-based choreography vendor Choreology for this article last month, it was a startup without any paying customers, despite two-and-a-half years of research and development, proof of concepts and standards championing. Hours later, the vendor had called back to report its first sale. "This changes our story somewhat," commented CEO Alastair Green with typically British restraint.
Once a rather esoteric concept, choreography is finally starting to find some resonance in the real world, urged on by credible standards and maturing product offerings. Furthermore, examples of the technology solving real business pain are starting to demonstrate the contribution it can make to the success of web services projects.
Choreography also known as orchestration is the layer in business process management (BPM) that takes multiple services and assembles them into business processes. Services can be components in transaction systems or application servers such as CORBA or J2EE; they can be XML web services called with UDDI, SOAP and WSDL; they can even be standard application services exposed through an adapter in fact, virtually any form of activity can be linked using web services interfaces, even faxes and emails.
Vendors either provide dedicated choreography servers such as the Collaxa Orchestration Server, Choreology's Cohesions 1.0 or the Intalio|n3 Business Process Management System, or they build orchestration functionality into existing server platforms such as Microsoft BizTalk Server 2004, BEA Weblogic and IBM WebSphere.
Examples of choreography in action show off its ability to create new composite applications by combining elements of existing IT systems into new processes. Aerospace and defense contractor BAE Systems wanted to extend the spare parts order management functionality in its IFS ERP system, in which many parts of the process were not automated, so that customers could place and monitor parts orders via a web portal. It used Intalio's Director product to change the user interface and add automation of the manual steps. This effectively upgraded the functionality of the IFS application, but without touching the underlying software.
In another example, FranceTelecom has set up a process hub that integrates and automates outsourced services supplied to its small and mid-sized business customers. While the project is still at an early stage, it is concentrating on vertical segments and has set up a first template of software and services for textile companies. It uses Collaxa to choreograph, change and update it.
The terms choreography and orchestration are effectively interchangeable, differing only in the scope of the co-ordination they encompass. Orchestration is more often used to refer to the flow of business processes within a single infrastructure and their interaction with other systems and organizations. Choreography is more often associated with collaboration between systems as opposed to internal workflow. Green at Choreology adds a third category, dubbed the "global view", which refers to a higher-level network of multiple collaborations "with complicated interrelationships".
Howard Smith, chairman of business process management industry body BPMI.org and CTO of IT professional services company CSC, says choreography or co-ordination as he calls it is the missing link that is often overlooked in web services projects. In his book, Business Process Management: The Third Wave, he puts the concept at the center of his definition of a business process: "A business process is the complete and dynamically co-ordinated set of collaborative and transactional activities that deliver value to customers."
The problem is that most web services projects stop once the services have been created, without going on to add the vital element of dynamic co-ordination between activities. "A lot of people are talking about taking SAP and existing systems and publishing them as services," says Collaxa's Khodabakchian. "[Choreography] goes to the next layer and says now you have all these services how can you manage them?"
Each layer needs the other, in fact. "It's a chicken and egg problem," says Khodabakchian. "The customer needs to be linked by a business process layer but it needs tools to take existing assets and expose them as web services too." In the absence of such tools, customers have either had to turn to expensive proprietary solutions or else do nothing, he adds: "A lot of people would either go and buy expensive EAI or reinvent middleware or they would sit on the fence and keep the processes manual."
Many therefore saw it as a turning point when enterprise software vendors SAP and Siebel joined with industry giants IBM and Microsoft earlier this year to back the BPEL4WS specification for orchestration of web services. Together with the companion specs WS-Coordination and WS-Transaction, BPEL is now the responsibility of e-business standards body OASIS, which also looks after BTP (Business Transaction Protocol).
The platform independence of BPEL helps companies insulate themselves from underlying platform differences, says Green: "What we can hope for is a cleaner, more uniform way of describing processes which will give people a common language, a more agile way of dealing with B2B interactions."
That in turn will help bring process co-ordination closer to the business user, vendors believe.
"One of the fundamental problems is the need to bridge the gap between the business user and the IT programmer," says Tom Barclay, senior product manager at Intalio. "A lot of these technologies have been around for a while but there has been no standard with rich enough semantics to map all business processes and do it graphically." Intalio's Xpage technology solves that problem, he says, by spanning seven languages, including Java and XML, so that analysts can model and rearrange processes irrespective of how each of them has been coded effectively divorcing the choreography from the programming layer.
Where transactions are involved, the stakes are high. "It's not easy to allow two companies to interoperate," says Green. "You need conformance testing, interoperability testing and you need to define trading protocols." In the past, companies have relied on building relationships of trust, but that's tricky to automate, he says. "Where you deal with people where there's enough trust, you're prepared to take a risk. But it's very difficult to get systems to go off and work together."
Nevetheless, companies are beginning to embrace choreography for its potential to enhance efficiency, reduce costs, or add value.
Many companies have traditionally spent huge amounts of time and money retrospectively correcting transaction errors, says Green, in areas such as telco provisioning and in long-running financial services transactions. The self-care web environment has made this reconciliation and repairs process much more visible to customers, he adds. "People spend months fixing problems after the fact because they've not had cleanliness up front," he explains. "They do the work, then inspect it, and then fix it. It's widespread. Our software enables co-ordinated outcomes, so we avoid or minimize post-transaction reconciliation and verification."
Intalio customer Lexis Nexis has used the technology to help it enter a new market among small and medium-sized law firms. Its existing order management systems were not cost-effective at this level of the market, because they had been designed with larger enterprises in mind. By using the Intalio|n3 BPMS, the legal information provider could still deploy functions from existing applications such as Siebel, while using them within more cost-effective processes created and managed in the Intalio platform.
Other customers aim to protect the intellectual property that's tied up in their process know-how, says Khodabakchian. "The number one reason customers are moving to business process standards is that their business processes are part of their IP. So they want to implement them in a standard way."
Most early adopters start out by testing business co-ordination software on a small scale. Vendors encourage this kind of incremental use of their products. "It's not like a new application," says Intalio's Barclay. "You can start with a limited scope project and then expand the scope once you are comfortable with the method."
For example, Intalio's first customer, iUniverse, started in just this way. The Nebraska-based self-publishing company has developed highly efficient, streamlined processes to help individuals bring their books to market. Since it didn't want to put those business-critical processes at risk with an untried technology, it started out by linking up non-core business processes. Following the success of these initial projects, the company now plans to run its entire business on the Intalio platform.
While the entry threshold is low, the possibilities are almost boundless, says Khodabakchian: "Any business process today that is manual or faxed or a phone call or e-mail can be orchestrated. Anything that's not exposed to the customer, you can take small steps to automate in a low cost and incremental way."
More on this topic
Windows brought drop-and-drag computing to the masses ...
Business process management is spawning a radical approach ...
Not only is BPEL4WS about to be handed to OASIS ...
Home page of the OASIS technical committee responsible for BPEL4WS
Website of the Business Process Management Initiative (BPMI.org)