Today Service Oriented Architecture (SOA) and Web Services are being proposed as the best solution for integrating multiple applications, the problem previously addressed by Enterprise Application Integration (EAI).

	express delivery
	• print  • comment
	As web services moves into the area of application integration, there is a need for a new integration layer, which navigates the differences in technology and information model of the applications. Ronan Bradley is CEO of integration software vendor PolarLake. The company's PolarLake JIntegrator product provides a J2EE-based Enterprise Service Bus with the ability to span multiple messaging oriented middleware (MOM) environments.
	Glossary terms: SOA, EAI, web services, composite application, ESB, lookup tool Sponsored links: your text link here

SOA is not enough on its own, however. It works best when it links up newly built, pluggable modules across a network to create distributed, composite applications. The enterprise integration problem is subtly different: it aims to connect together pre-existing, self-sufficient applications, which don't necessarily conform to the assumptions of SOA's distributed application model:

• SOA assumes there is a common technology stack shared between the consumer and the provider of the service, required to allow the modules to interact. Even with open standards such as web services standards, there can be differences in implementation that cause problems.

• The consumer must be written, or modified, to interact with the service definition. This is fine for new application development or when the consumer is a small client-side application. However, in many cases the applications that must be integrated are existing monolithic applications that cannot be easily changed.

These problems are manageable or even irrelevant if we are developing new clients to connect to existing servers, or can guarantee the technology stack will be standard across the enterprise.

Unfortunately, neither is typically true. Most projects revolve around the integration of existing systems and technologies, which work just fine and cannot be changed. A company exclusively relying on a single technology stack is also rare (and even then multiple, often incompatible, versions of the same software are used).

These issues are beginning to become more generally understood. In a recent research piece (Most Composite Applications Will Need an Integration Layer, April 2003), Gartner analysts Roy Schulte and Yefim Natis explain this problem in terms of a technology mismatch and an information model mismatch. They go further to say that there are only two ways to solve these two mismatches:

• Insert a layer of integration logic between the client and server modules.

• Build the client module to conform exactly to the characteristics of every server module that it will invoke.

The second approach is unacceptable in many cases because it forces changes into existing applications and over time turns each integrated application into a spaghetti of multiple adapters for each server that the system needs to connect to. This leaves the first approach as the only viable alternative: a separate integration layer which acts as mediator between the applications. The rest of this article focuses on the characteristics the integration layer should have.

The technology gap

Another way of looking at the technology gap is to say that the integration layer must coexist with deployed infrastructure and applications, not attempt to replace them.

Things to check for in an integration-layer solution (you will probably need most but not all of them in any given project):

• Does it work with the existing messaging systems (IBM WebSphere MQ et al) as well as with Internet protocols?
• Does it work with the existing management?
• Will it support the typical integration modes such as:
  
  • Interface driven, both application server-based and common applications such as SAP?
  • Data driven, integrating with relational databases and real time feeds?
  • User interface driven, integrating with screen-driven mainframe systems?
  
  
• Does it support distributed management and deployment to allow the software to be controlled across the network?
• Is it going to meet performance and scalability criteria?
• Does it support the common routing model, such as publish-and-subscribe or content-based routing?

Many of these requirements are included within the definition of an Enterprise Service Bus (ESB).

The information model gap

So we now have a layer that can push and pull SOAP and XML documents around. The next problem we encounter is 'the information model gap'.

When an XML or SOAP document arrives at an application, there may still be a gap between that document and the semantic and process model of the application. The problem may be as simple as the format being wrong or as complex as the incoming document being passed through a complete process before the application can accept it (perhaps validating the sender, gathering secondary information from other systems and then transforming the information into the appropriate request).

Our experience has shown that crossing the gap requires three types of activity:

• Validation against business definitions. Accepting an invalid document has potentially disastrous consequences for any application, and unfortunately it is easy to create XML that is technically correct and will pass any schema validation but is nonetheless nonsense from an application perspective.

• Enrichment with additional information (which can be close to a nested BPM). This ensures the data is complete, and is likely to involve other applications (eg databases, web services, etc) as well as internal calculations (eg aggregation, derivation, analysis, etc).

• Transformation into the correct format before being mapped into the Java domain. There might be more information that must be filtered out, or less information that must be augmented in some way, or the information may be in a different format or order.

In addition:

• Exception handling must be managed intelligently.

Strong validation, enrichment and transformation capabilities are the first three attributes to be checked in any potential solution. This last attribute, exception handling, is probably the least obvious, even though it was reported by the Hurwitz Group in July 2001 that nearly 80% of the time spent in building business processes is spent in exception management.

Conclusion

As web services moves into the area of application integration, there is a need for a new type of solution: a new integration layer. This integration layer navigates the differences in technology and information model of the applications. To finish with a quote from Gartner again:

"In the near future, most composite service-oriented architecture will use an integration layer."

More on this topic