By ^Author

^Abstract

^Sidebars

Enterprise Application Integration (EAI) is really nothing more than putting science, methodology, and technology behind the integration of information systems. As we move into a more connected world, application integration in support of mission-critical concepts such as business-to-business (B2B) integration, drives EAI, as does the availability of modish and more effective middleware technology for use with the Internet.

However, EAI remains a difficult concept to grasp for most organizations, and an even more difficult concept to successfully implement. The barriers to application integration still exist, including incompatible application interfaces or lack thereof, limitations of current middleware technology, and the expense of adapting your entire enterprise to EAI.

While there are clear challenges to application integration, the need to bind applications together both within and between enterprises is driving the interest in EAI as well as the new technology offerings to support it. Message brokers, unheard of just a few years ago, are now dominating most EAI solution sets, as are Java-enabled application servers, and even traditional middleware such as message-queuing software.

Clearly, the essence of EAI is something we've been trying to achieve for years. Now that B2B integration and the exposure of existing enterprise systems to the Web have become imperative, EAI can no longer wait. Considering this, it's time to better understand the ins and outs of application integration, finding the sweet spot in approach and technology for your particular organization. As always, to better understand a problem and its solutions, we need to break it down into its components.

We can break application integration into macro problem domains: intra-enterprise, or application integration that occurs within the firewall (traditional EAI); and inter-enterprise, or application integration that occurs between two or more distinct organizational entities (e.g., companies, agencies, etc.). We can further break each macro problem domain into application integration types, which include data level, application interface level, method level, user interface level, and information portals.

Intra-Enterprise Integration

Intra-enterprise application integration is the joining of systems that serve a single enterprise. Typically, this means linking the older mainframe-based inventory system, written in DB2 and Cobol, with the SAP R/3 system just installed, or perhaps linking both systems with the sales automation systems of a company just acquired.

In the past, companies addressed most intra-enterprise application integration requirements through whatever means they could find. This included nightly or weekly batch updates, FTP file transfers, and rekeying data. Later, they learned to join systems together using traditional point-to-point middleware products such as the Open Software Foundation's DCE (Distributed Computing Environment) or IBM's MQSeries message queuing software.

The point-to-point nature of these products, while providing the ability to share information between a single source and a single target system, quickly became difficult to manage and constantly required expensive changes to all participating systems. In short, leveraging traditional point-to-point middleware could not scale to the massive application integration needs today's businesses are encountering.

Today the answer is leaning toward more solutions-oriented and strategic approaches and technology, including creating a common infrastructure for enterprises to share information in one-to-one, one-to-many, and many-to-many configurations (see "Integration Infrastructure"). Newer, but less-proven technology, such as message brokers and application servers, are replacing more traditional point-to-point middleware products and ad hoc integration solutions. Albeit, hyperbole seems to be leading reality as we once again consider the latest magic bullets. As time pushes on, we have a clearer view of what these new products will and won't do for us.

Integration Infrastructure

Scaling up. Point-to-point integration solutions cannot scale up to today's massive integration needs. Instead, enterprises are leaning toward a common infrastructure for sharing information.

Types of EAI

When contemplating intra-EAI, an organization must first understand the sum and content of the business processes and data in the organization. IT also needs to understand how these business processes are automated (or not), and the importance of all business processes. Depending on the enterprise, this may demand a significant amount of time and energy. In brief, organizations must understand both business processes and data. They must select which processes and data elements require integration. The types of EAI solution sets can take on several dimensions.

Types of EAI

Which type do you need? EAI products come in many flavors, and offer integration at various (and sometimes overlapping) levels.

Data-level EAI is the process -- and the techniques and technology -- of moving data between data stores. This can be described as extracting information from one database, perhaps processing that information as needed, and updating it in another database. While this sounds direct and straightforward, in a typical EAI-enabled enterprise it might mean drawing from as many as 100 databases and several thousands of tables. It may also include the transformation and application of business logic to the data that is being extracted and loaded.

The advantage of data-level EAI is cost. This approach largely leaves the application alone and does not change code, so organizations don't need to incur the expense of changing, testing, and deploying the application. What's more, the technology that provides mechanisms to move data between databases as well as reformat that information, is relatively inexpensive.

Application interface-level EAI refers to the leveraging of interfaces exposed by custom or packaged applications. Developers leverage these interfaces to access both business processes and simple information. Using these interfaces, developers can bundle many applications together, allowing them to share business logic and information. The only limitations that developers face are the specific features and functions of the application interfaces.

This type of EAI is most applicable to packaged applications such as SAP, PeopleSoft, and Baan, which all expose interfaces into their processes and data, but do so in very different ways. In order to integrate those systems with others in the enterprise, developers must use these interfaces to access both processes and data, extract the information, place it in a format understandable by the target application, and transmit the information.

Method-level EAI is the sharing of the business logic that may exist within the enterprise, or between enterprises. For example, the method for updating a customer record may be accessed from any number of applications by invoking a common shared method, typically residing on a shared application server or distributed object infrastructure. By sharing methods, applications are tightly coupled and therefore tightly integrated.

The mechanisms to share methods among applications are numerous, including distributed objects, application servers, TP (transaction processing) monitors, frameworks, and simply creating a new application that's the combination of two or more applications.

There are two basic approaches: Organizations may create a shared set of application services (methods) that exist on a shared physical server, such as an application server. Or, methods already existing inside of applications can be shared using distributed method-sharing technology such as distributed objects ("wrapping" applications to expose methods).

Method-level EAI is something companies have been practicing for years as they've sought to reuse application development efforts within the enterprises. This approach hasn't been very successful, due to both human and technological issues. Perhaps with EAI, we may get it right.

User interface-level EAI is a more primitive, but nonetheless necessary, approach. Using this scenario, architects and developers can integrate applications by using their user interfaces as a common point of integration (also known as screen scraping). For example, mainframe applications that do not provide database- or business-process-level access may be accessed through the user interface of the application.

Although many consider leveraging the user interface as a point of integration to be an unstable and archaic approach, the fact is that it has been done this way for years and many of the issues, such as performance, reliability, and scalability, have been worked out. Although not preferred, it may be the only solution in many instances. The heart of EAI is the ability to leverage any existing system by finding a reliable point-of-integration.

The Information Portal type of EAI is very popular today due to the rise of the Web. Using this EAI approach, application architects can integrate applications by presenting information from several applications within the same user interface.

Those of us who use Web portals such as excite.com are familiar with this concept. Information from many places, such as other sites or applications, is presented within the same user interface, typically a Web browser. Enterprises are leveraging this integration approach as a means of integrating enterprise systems (such as inventory, SAP, and sales automation systems from the example above) at the user interfaces, thus avoiding the complexity and expense of traditional back-end integration.

Inter-Enterprise Integration (E-business)

The mechanism, techniques, and technology that make up EAI are also applicable to most e-business scenarios, or inter-EAI. Because EAI, as a concept and technology suite, is very good at integrating various applications and data stores, it can extend its reach outside the enterprise to include both trading partners and customers within the application integration architecture. This enables any application or data store to share information with any other application or data store that exists within the supply chain scenario.

Traditional B2B integration scenarios leveraged traditional technology such as EDI (electronic data interchange) and proprietary value-added networks (VANs). Today, however, most B2B architects opt for more realtime and Internet-enabled technology such as Internet-aware message brokers and application servers, as well as new data interchange standards such as XML.

Using newer application integration mechanisms and technology allows companies to link, for example, their SAP system to their supplier's Baan system and move information between them as needed to support the B2B integration scenario. Companies may also include a "dealer" system in the supply chain by using application integration technology and techniques to first create the links and then the methods to share both data and business processes, using the Internet as a common public network in many cases.

EAI is the "missing link" in the quest for success in B2B integration efforts of the past. There has always been the desire to have systems communicate seamlessly and in real time, with suppliers' systems, but until now inexpensive solutions that could make it happen have not existed. The new generation of middleware that supports EAI concepts is making this a reality.

As in B2B integration, B2C (business-to-consumer) EAI exposes information within an enterprise to people or entities that exist outside the enterprise. For example, systems must be integrated before externalizing the information as a virtual system in support of Web-based selling and information sharing efforts. It's clear that B2B and B2C are the new drivers of EAI approaches and technology.

Enabling Technology

No matter what EAI type a company selects for its problem domain, the proper enabling technology, typically in the form of middleware, must also be selected. While there are many types and brands of middleware, the three types that are having the most impact on both intra- and inter-application integration include application servers, distributed objects, and messaging brokers.

Application servers not only provide a location for application logic, but they also coordinate many resource connections using transactional semantics. This enables users of application servers to integrate back-end resources using the notion of a transaction (start, extract information, move information, update information, stop), as well as expose information from those resources to connected Web browsers or through client/server interfaces.

Application servers typically provide application development features such as an IDE (integration development environment), as well as application testing and deployment facilities. The new application servers tend to support Java as both the native language and the development model (although in different ways) as they move toward the world of Enterprise JavaBeans (EJBs).

Application servers provide an up-to-date approach to sharing methods using the power of transactions, thus supporting method-level EAI, as well as a mechanism for integration. However, each application server has its own approach to how this happens. Therefore, making wise decisions regarding the selection of an application server requires an understanding of the category of features, as well as the problem at hand.

Distributed objects are considered middleware because they facilitate inter-application communications. However, they are also mechanisms for application development (therein lies an example of the middleware paradox), providing enabling technology for enterprise-wide method sharing, thus supporting method-level EAI.

Distributed objects are really small application programs that use standard interfaces and protocols to communicate with one another.

For example, developers may create a CORBA-compliant distributed object that runs on a Unix server and another CORBA-compliant distributed object that runs on an NT server. Since both objects are created using a standard (in this case, CORBA), and both objects use a standard communications protocol (in this case, Internet Inter-ORB Protocol--IIOP), they should be able to exchange information and carry out application functions by invoking each other's methods. Thus, distributed objects provide coupled application integration features.

Message brokers represent the nirvana of EAI-enabled middleware. At least that's the way they are being sold. Message brokers can facilitate information movement between two or more resources (source or target applications) and can account for the differences in application semantics and platforms. As such, they are perfect matches for EAI. Message brokers can also join many applications using common rules and routing engines. They can transform the schema and content of the information as it flows between various applications and databases.

Message brokers are servers that broker messages between two or more source or target applications. In addition to brokering messages, they transform message schemas and alter the content of the messages so the messages make sense to the application receiving the message.

In contrast to application servers, message brokers tend to move information using event-driven mechanisms. Message brokers watch for state changes at the source or target systems (e.g., update to a customer database) and react accordingly, typically by moving and reformatting business information. A single state change could set off hundreds of events --a chain reaction of information movement and business event handling -- inside a message broker or the connected applications. Application servers, by contrast, use a transactional model where transactions are invoked within the application server. The application server carries out a preprogrammed function, perhaps communicating with an application resource. The message broker model is loosely coupled to its connected applications where the application server is tightly coupled. Loosely coupled solutions typically don't require changes to source or target applications, and are least expensive to implement. Selection of an application server or message broker depends on your problem domain.

The importance of message brokers is a function of their place within the enterprise. Message brokers are not an application development-enabling technology. Rather, they are a technology that allows many applications to communicate with one another without actually understanding anything about each other. In short, they "broker" information.

Application Integration - Just Getting Started

Application integration, both in the context of EAI and B2B, is becoming a hot topic. We finally understand that information systems that communicate bring clear value. But it's a long way from simply understanding the value to realizing it on the bottom line. It's going to take years of work before we are even a few steps closer to having a fully integrated enterprise, not to mention a fully integrated B2B solution set. Planning, architecture, and learning need to be on the top of every architect's list right now.

The technology and understanding available today bring much more to the table for those interested in solving this problem. Those who are willing to view application integration as a strategic versus a tactical play will ultimately succeed. We are moving to an event-driven realtime economy. Those who embrace it will survive. Those who don't will fall by the wayside. It's really just that simple.

David S. Linthicum is the CTO of SAGA Software in Reston Va., and the author of Enterprise Application Integration (Addison Wesley Longman). E-mail him at linthicum@worldnet.att.net.