Integration Complexity
This business of running a business has become a lot more complicated and much of it can be attributed to the use of information technology. Technology has indeed fueled growth through automation and increased efficiency, but it has also added a management nightmare in the form of disparate systems that are unable to talk to each other. Now we have whole new specialties (like ITIL and COBIT) devoted to managing IT services, assets and infrastructures. How did we end up in this quagmire? Each department was given a carte blanche to buy systems that were most suitable for their operations. Such best-of-breed purchases resulted in a proliferation of applications that could not connect with each other.
Then the integration vendors stepped in to implement point-to-point integration solutions, which turned out to be so expensive that only the large companies could afford. Smaller companies were left to live with the problem.
Enterprise application vendors on the other hand, created a vision of a single application and pushed for that across the enterprise. While that sounded good on paper, it did not work well for the customer due to vendor lock-in, the vendor’s inability to provide all the components, the high cost, and the “templatization” of the process causing customers to lose their competitive advantage.
How To Fix The Problem
How does a modern enterprise fix this disparate application communication problem? There are four basic strategies: (a) outsource, (b) rip-and-replace, (c) do a point-to-point integration, or (d) implement a service-oriented architecture.
Outsource
The outsourcing option is most suitable for business processes that are not the core competence of the organization. For instance, many companies outsource payroll management, a non-core process, with good success. On the other hand, core processes that help to establish a competitive advantage, like Dell’s supply chain, is best kept within.
Outsourcing does not imply abdicating responsibility. JP Morgan Chase’s $5 billion IT operations outsourcing plan to IBM, for instance, had to be brought back in for a multitude of reasons. However, when done judiciously and managed well, outsourcing can be a boon.
Rip-And-Replace
In this approach a business completely revamps its operations by ripping out and replacing the relevant applications, with the hope of increasing operational efficiency. Companies have typically shied away from this approach due to committed investments, high cost, time, and practical implementation difficulties. Unless the processes and applications are in complete disarray, this kind of radical approach will not provide a good ROI.
Point-To-Point Integration
As stated earlier, point-to-point integration to connect proprietary systems becomes expensive, both in terms of time and money if it has to be done on a large scale. However, it may work well for smaller integration projects. If the long-term business and strategic vision calls for a large number of system integrations, this approach is not scalable.
Service Oriented Architecture (SOA)
In this approach, application functionality is exposed as services that can be consumed by applications. In a web-based environment, web-service integration is effected by the orderly exchange of data using XML files over a standard protocol such as SOAP (Simple Object Access Protocol). This order is usually determined by an operational business process. However, one needs to ensure that the implementation is driven from the business side and not driven from the IT side by assembling components.
BPM And SOA
The disciplines of Business Process Management (BPM) and SOA are often touted as the best approach for application integration. While this approach helps businesses achieve operational efficiency, better governance, agility and control, the ramifications are often ignored. The goal of this article is to further explore three technical areas.
An SOA Driven Implementation
An effective BPM/SOA strategy has to be driven from the business side. Unfortunately, in many businesses, as it is easier for the IT department to catalog a list of all the services by application, it is usually driven bottom-up. This will not only result in proliferation of services, but more importantly, may fail to address business needs. Therefore it is critical for the organization to establish a framework that drives the need for services that need to be encapsulated and callable, rather than end up with a proliferation of useless and often duplicate services that becomes a management nightmare.
Distributed Data
In a SOA architecture, the data is distributed across different applications with no effort at consolidation. Duplicate, missing, and inconsistent data are reconciled through a single service interface. For example an update to a customer’s address may actually involve updating multiple databases. There is no problem when the service implementation works well. However, if the transaction fails to update the record in one database, then the databases can be left in an inconsistent state.
While an ideal solution to this would be to roll-back the transaction, that is often not possible. In such cases a compensating transaction needs to occur. For instance, if the update of the customer address to one database prompted a letter to be printed out and sent to the customer as a confirmation, then a compensating activity would involve sending another letter stating that the first one was a mistake. Inevitably, this does not project a good impression to the customer.
More often than not, such exception conditions are not programmed into the system, primarily because designers and programmers often think about these exception conditions as an afterthought. Exception handling should be treated with the same rigor as the “normal” processing path.
Perception Of Agility
The BPM/SOA approach is often sold as an agile approach wherein changes to business process can be quickly incorporated into the system. This statement holds true only in a limited number of cases and simple changes. For example, if in a process a manager’s approval is required for any PO that exceeds, say, $500, and we want to change that to $2000 instead, then that’s a trivial change that can be easily accommodated.
On the other hand, if the change requires an approval from an accountant as well, then adding a new screen with different information that has to be pulled in from different systems is relatively non-trivial. This will call for screen design, data integration, XML document definition, and so on.
Conclusions
While BPM/SOA is often touted as the best approach to increase operational efficiency, achieve better governance, and make the business agile, the ramifications are often ignored. In this article, we delved further into three specific aspects to showcase the complexity of the implementation.
