Introduction
Organizations have been using numerous applications that were built over many years. These applications probably are of different era, were written by using different languages and technologies, reside on different hardware platforms, use different operating systems, and provide very different functionality. In fact, many of the applications often have very little in common at all, resulting in isolated functionality and multiple instances of the same data. For your organization, these conditions can result in redundant activities, higher costs, and inefficient response to your customers.
Types of Application Integration
Application integration can be broadly categorized into three types:
· Manual application integration
· Semi-automated application integration
· Fully automated application integration
Application Integration type
| Cost
| Benefits
|
Manual
| Higher labor costs that scale badly. Subject to human error.
| Little change required from existing low-technology environment.
|
Semi-automated
| Higher technology costs to implement. Subject to design-time and run-time errors.
| Lower labor costs. Scales better. Less subject to human error at run time. Faster processing.
|
Fully automated
| Highest technology costs to implement. Subject to design errors.
| Lowest labor costs. Not subject to human error at run time. Lose human decision making on business processes, but faster processing.
|
Making Application Integration Scalable
An important part of making application integration scalable is to increase the number of automated steps and reduce the number of human steps. This generally involves creating interfaces between applications along with predefined logic that replaces human involvement. For automated application integration, you have two choices:
● Point-to-point model
● Integration hub model
Requirements for Application Integration
Typically, an environment that supports application integration meets at least the following requirements:
· Connectivity between different platforms
· Processing of complex business rules, including complex data transformation logic
· Support for business processes, from the very short to the very long, including processes that last weeks or months as data is passed and processed through different parts of the organization
· The ability to modify existing business processes or create new ones as business goals change
· The ability to adapt to changes in hardware, software, and business goals
To help meet these requirements, your application integration environment should:
· Expose a common interface through which applications can communicate, by using business semantics to request Web services.
· Allow service requests at the functional or data level for applications that do not support using business semantics.
· Use a common set of process and service rules to ensure consistency and reuse of integration services.
· Be capable of reusing the existing transport protocols that already exist in the enterprise.
· Insulate it from existing technologies by using interfaces.
Challenges of Application Integration
Technical Issues
When building an application integration environment, you are creating an environment that allows components to communicate with each other even though they were never designed to do so. In many cases, these components are not even aware of each other. They may include a variety of preexisting systems (some of which are essential), yet your environment needs to be noninvasive to existing applications.
In these conditions, complex architectural issues such as the following can arise:
· Control and coupling
· Data exchange and data formats
· Distribution and concurrency
· Scalability, reliability, and availability
You do not want to completely rebuild the application integration environment itself every few months. The aim, instead, is to have a relatively stable application integration environment, which is flexible enough to allow the addition of new applications or the alteration of relationships between applications.
Organizational Issues
Organizational issues in application integration arise mainly from the fact that the application integration environment will span multiple departments in the organization. Staff in different departments may choose to deploy applications that will then need to integrate with the rest of organization environment. To design a truly effective application integration environment, you need to isolate application integration as a separate management function, and determine who is responsible for making it happen. In larger organizations, you may have a group of people specifically responsible for this function, including application integration architects, application architects, and application owners, whereas in other cases, application integration may be one of many management functions that you are responsible for. Either way, by defining the application integration management function in this way, you help to ensure that it is properly considered whenever changes are made to the IT infrastructure.
Levels of Application Integration
Application integration can occur at many different levels, including:
· Presentation level
· Business process level
· Data level
· Communications level
Business Process – Level Integration
Business process – level integration often forms the starting point for defining application integration environment, because it is most closely related to the business requirements of the organization. Business process – level integration starts with defining your business processes and then specifying the logical integration that corresponds to those processes.
Defining Your Business Processes
Before you can design your application integration environment, it is important to understand the organization in logical terms, removed from the technology that underlies it. Business processes consist of a group of logical activities that model the activities in your organization. These activities typically represent the smallest units of logical work that can be done in the organization.
Mapping Business Requirements to Application Integration Requirements
After you have determined the business processes that the organization uses, you can begin to examine how they affect your application integration requirements. It is important to realize that a simple business process does not always correspond to simple requirements for application integration.
By modeling each of your business processes, you can define which applications must integrate with one another, and in what way. This enables you to make intelligent decisions about the specifics of application integration environment.
Data-Level Integration
You can define how applications communicate with each other at a business process level, but if they cannot understand the data they exchange, they cannot integrate successfully
There are two general ways to enable data-level integration:
· Add logic to enable each application to understand incoming data from other applications.
· Add logic to enable each application to interpret outgoing data to an intermediate data format and interpret incoming data from that format into a form the application understands.
To support the various complex data integration requirements, your application integration solution must often contain a considerable amount of logic that supports the access, interpretation, and transformation of data. You also need schematic models of enterprise data to describe data attributes. The definition and recognition of schemas enables the validation of data. Descriptions of how data elements may be related or mapped to each other allow for the transformation of data.
Communications-Level Integration
Enabling Communication between Applications
Because many applications are not designed to communicate directly with each other, it is likely that you will need to do some development work to ensure that your applications can be called by other applications.
There are two possible approaches to this problem:
· Rewrite your application to provide it with an API that other applications can call.
· Create a communications adaptor that acts as an intermediary between the application and other applications.
Important Considerations for Application Integration
There are few important considerations regarding Application integration and they are as follows
· Using Synchronous and Asynchronous Communication
· Increasing Automation
· Straight-Through Processing (STP)
· Ensuring Data Integrity
· Managing Latency
· Data Aggregation
· Combining Application Functionality
· Application Composition
· Managing Transactions
o Providing ACID Transactions
o Transactional Messaging
o Two-Phase Commit
o Long-Running Transactions
o Timed Transactions
Security and Operational Issue
Security Issue
Because different applications have different security requirements and features, it can be quite a challenge to ensure that your application integration environment functions properly without compromising your security requirements. Security is particularly important in application integration because a breach in an integration service may result in security breaches in other integrated systems.
Security Policy
The first step toward effective security in any environment is creating a written security policy. Many factors can affect the security policy, including the value of the assets you are protecting, the threats that your environment faces, and the vulnerabilities that are currently present
From an integration standpoint, your security policy should define:
- A mechanism for evaluating and classifying threats
- A mechanism for acting on threats
- A boundary for information security
- A plan for communication and enforcement.
- General security guidelines
- Reference to other documents
- A mechanism for modifying security policies
Defining Your Security Requirements
The precise security requirements of application integration environment depend on a number of factors, including the following:
· Security requirements of the organization
· Business requirements for application integration
· Technical requirements for application integration
· Capabilities of the applications which will be integrated
· Platforms of the applications
· Budgetary constraints
As a starting point for defining your security requirements, you should perform a risk analysis. Doing so allows you to identify the threats and vulnerabilities that you face and identify the countermeasures you can deploy to keep risk at an appropriate level.
Operational Management Considerations
Even after you build an application integration environment and it is running successfully, your work does not stop there. The environment will need to be monitored and maintained over time. This section discusses the various operational management considerations involved. Implementing the capabilities listed in this section should help to ensure that your architecture continues to function as smoothly as possible.
Defining an Operational Management Policy
Your operational management policy, at the minimum, should provide the following information:
· Clearly defined terminologies and target metrics
· Methodology and/or formulas for measuring the metrics to ensure that results are consistent
· Service-level prioritization to ensure that the most important policies are followed first
Defining Operational Requirements
The operational requirements of application integration environment depend on a number of factors, including the following:
· System Monitoring
· System and application health
· System and Application Performance Monitoring
· Security monitoring
· Service-level monitoring
· Business Activity Management
· Business transaction exception handling
· Contextual monitoring
· Rules-based alerting
· Historical data mining
· Change and Configuration Management
· Directory
Microsoft Technology/Tools Mapping on EAI Capabilities
Capabilities Required for EAI Environment
The capabilities are defined as follows:
· Business process integration capabilities
· Data integration capabilities
· Communications capabilities
· Security capabilities
· Operational management capabilities
Business Process Integration Capabilities
Capability
| Provided By
|
Rules Processing
| Custom .NET Framework components, SQL Server stored procedures, and BizTalk Server Business Rules Engine
|
Transaction Management and Compensation
| Microsoft Distributed Transaction Coordinator (DTC) and BizTalk Server long-running transactions
|
Workflow
| Windows Workflow Foundation and BizTalk Server Human Workflow Services
|
Orchestration
| BizTalk Orchestration service
|
State Management
| Common language runtime, IIS session management, and BizTalk Server orchestration state management
|
Event Processing
| SQL Server triggers, MSMQ Events, and BizTalk Server messaging subscriptions
|
Schedule
| Windows Server Schedule service and SQL Server Job Scheduler
|
Contract Management
| Windows Communication Foundation and BizTalk Orchestration service
|
Data Integration Capabilities
Capability
| Provided By
|
Data Validation
| XML schemas
|
Data Access
| ADO.NET and SQLXML
|
Schema Definition
| Visual studio 2003/2005
|
Data Transformation
| SQL Server DTS, XSLT and BizTalk Mapped
|
Schema Recognition
| XML schemas and BizTalk Server parsers
|
Communication Capabilities
Capability
| Provided By
|
Message Routing
| Windows Communication Foundation and BizTalk Server Message Box and subscriptions
|
Message Delivery
| BizTalk Server message ports
|
Message Queuing
| Microsoft Message Queuing and BizTalk Message Queuing
|
Message Forwarding
| Windows Communication Foundation and BizTalk Server Message Box and subscriptions
|
Message Correlation
| BizTalk Orchestration correlation services
|
Message Instrumentation
| BizTalk Orchestration
|
Addressing
| BizTalk Server message ports
|
Transactional Delivery
| Microsoft Message Queuing and BizTalk Server reliable messaging
|
Security Capabilities
Capability
| Provided By
|
Authorization
| Windows Card Space and Active Directory
|
Authentication
| Windows Card Space and Active Directory and Share Point Portal Server Enterprise single sign-on
|
Information Protection
| Windows Server
|
Identity Management
| Windows Card Space and <city w:st="on"><place w:st="on">Enterprise single sign-on
|
No repudiation
| Digital signatures
|
Profile Management
| <city w:st="on"><place w:st="on">Enterprise single sign-on
|
Security Context Management
| Active Directory and <city w:st="on"><place w:st="on">Enterprise single sign-on
|
Operational Management Capabilities
Capability
| Provided By
|
Business Activity Management
| BizTalk Server Business Activity Monitoring (BAM)
|
Event Handling
| Windows Management Instrumentation (WMI) events, Event Viewer, Performance Monitor, and MOM
|
Configuration Management
| System Management Server
|
Directory
| Active Directory
|
Change Management
| BizTalk Server Business Rules Engine and schema versioning
|
System Monitoring
| Windows Server and MOM
|
References
http://msdn.microsoft.com/practices/Topics/appint/default.aspx?pull=/library/en-us/dnpag/html/eappint.asp
http://msdn.microsoft.com/practices/Topics/appint/default.aspx?pull=/library/en-us/dnpag/html/intpatt.asp
http://www.developers.net/intelisdshowcase/view/840
http://www.infoq.com/j+n/
