SOAD an introduction - I

There has been a lot of buzz and hype surrounding the opportunities presented by Service Oriented architecture (SOA) and its implementation as webservices. For many newbies SOA is all about service enabling components to some form of webservice and publishing them using standards. In reality, SOA is not about products and standards that help realize it, for example through webservice, but beyond those like services, their flows, compositions, quality of service, and governance. This brings us to service oriented modelling which is quite different from object oriented modelling in the sense that the later is more use case driven whereas the former is more process driven. In most cases, service oriented modelling is a hybrid of business process modelling, object oriented modelling and enterprise architecture.

To work on SOA projects, one may have to analyze, design and make architectural decisions at various layers like operational system or legacy layer, Enterprise component layer, Services layer, Business process or orchestration layer and the Integration layer.

Legacy layer : This consists of existing custom built applications, including existing CRM and ERP packaged applications, and older object-oriented system implementations, as well as business intelligence applications. The composite layered architecture of an SOA can leverage existing systems and integrate them using service-oriented integration techniques, which is a common case in most businesses.

Enterprise components layer: This is the layer of enterprise components that are responsible for realizing functionality and maintaining the QoS of the exposed services. This layer typically uses container-based technologies such as application servers to implement the components, workload management, high-availability, and load balancing. For example, components in these layers could be enterprise beans, asynch beans, message queues, thread pools, resource adapters, etc.,

Services layer: The services the business chooses to expose reside in this layer. They can be discovered or be statically bound and then invoked, or even, choreographed into a composite service. This service exposure layer also provides for the mechanism to take enterprise wide components, business unit specific components, and in some cases, project-specific components, and externalizes a subset of their interfaces in the form of service descriptions. Thus, the enterprise components provide service realization at runtime using the functionality provided by their interfaces. The interfaces get exported out as service descriptions in this layer, where they are exposed for use. They can exist in isolation or as a composite service.

Business process composition or choreography layer: Compositions and choreographies of services exposed in Service layer are defined in this layer. Services are bundled into a flow through orchestration or choreography, and thus act together as a single application. These applications support specific use cases and business processes.

Integration (ESB) Layer: This layer enables the integration of services through the introduction of a reliable set of capabilities, such as intelligent routing, protocol mediation, and other transformation mechanisms, often described as the ESB (the EDI equivalent). Web Services Description Language (WSDL) specifies a binding, which implies a location where the service is provided. On the other hand, an ESB provides a location independent mechanism for integration.

QoS Layer: This layer provides the capabilities required to monitor, manage, and maintain QoS such as performance, reliability, transaction, and availability. Some of these attributes are implemented at the Enterprise component layer (like transaction, security) whereas the others are monitored via background process that check the health of SOA applications.

              As it can be seen now, design strategy for a SOA does not start from bottom-up as is often the case with a Web services-based approach. SOA is more strategic and business-aligned, Web services are a tactical implementation of SOA. A number of important activities and decisions exist that influence not just integration architecture but enterprise and application architectures as well.

What is Reliability?

When architects or leads present one solution vs other one, they often speak of performance and throughput but quite ignore most of the other software attributes like maintainability, manageablity, availability, and reliability until the first failure. To accomplish each of these attributes in your sofware you need to follow certain design principles and build into your design rather hoping for it to be taken care at deployment time, though some of these are deployment attributes. In this post, I will discuss about the reliability factor which often is linked to availability and measured in 9's, for example a system is 99.9% available meaning the probability of system failure is 0.1 or 10%. So, how do one measure reliability. To understand, one need to agree on the definition of reliability.

There are actually two components that impact the reliability of a system

• How long it will work before it fails
• How long it will take to fix it.

We call the first the mean time before failure (MTBF) and the second the mean time to repair (MTR). In large transaction processing systems, reliability is often measured as down time. Down time has a cost associated with it – perhaps $1,000 per hour or $100,000 per hour. (Of course, MTR plays a role here as well, as the longer the down time, the higher the cost in many cases – from customer annoyance to lost sales to lost customers.) Down time alternatively can be measured as the proportion of time that a system is up, a measure that we call the availability of the system. Since the system is always either up or down, availability can be defined as,

A = MTBF/ MTBF + MTR

One can also define,

Availability = Doing the right thing = A

Failure = Doing the wrong thing = F = (1-A)

Reliability = Not doing the wrong thing = 1/(1-A)

To increase availability, the most intutive approach is to add redundant systems with full replication. However, depending on number of spares introduced in your topology, this introduces additional cost and replication issues. In general, more the number of nodes higher the chances of data collision, where users from two different systems try to update the same data. Another approach is to split the system so that under failure the other node assumes full load. Failure probability of this configuration is now,

F = n/k*(n/k-1)*(1-a)*(1-a)/2

where 'n' is number of elements or applications that run in that system and 'k' the number of nodes.

Rule of Thumb:

1. If a system is split into k parts, the resulting system network will be more than k times as reliable as the original system and still will deliver k/k-1 of the system capacity in the event of an outage.
2. If a system is split into k parts, the chance of losing more than 1/k of its capacity is many, many times less than the chance that the single system will lose all of its capacity.

JavaScript Threads

Most web developers must be aware (hopefully), that javascripts are executed on a single threaded environment within browser frame or tab process. At render time your scripts are executed sequentially and have to wait or yield until the previous request is complete. Many might have even noticed browser dialog

To mimic concurrency, developers use techniques like setTimeout(), setInterval(), XMLHttpRequest, and event handlers. All of these features run asynchronously, but non-blocking doesn't necessarily mean concurrency. Asynchronous events are processed after the current executing script has yielded. This is the primary reason why some page onload or javascript controlled behaviors for page rendering work most of the times but not always. Often these issues are blamed to slow network because you may not notice such behaviors every day.

The good news is HTML5 has put an end to this bringing threading into javascript world. Using Web workers, developers can now run heavy weight scripts in their own threads without blocking other components in the page. W3C specifications talks about two types of workers, shared and dedicated.

Dedicated Web Workers run in an isolated thread. As a result, the code that they execute needs to be contained in a separate file. But before we do that, the first thing to do is create a new Worker object in your main page. The constructor takes the name of the worker script:

var worker = new Worker('task.js');

If the specified file exists, the browser will spawn a new worker thread, which is downloaded asynchronously. The worker will not begin until the file has completely downloaded and executed. If the path to your worker returns an 404, the worker will fail silently. As with java threading, javascript threads do not return values or allow you to trap failures.

After instantiating the worker, you start it by calling the postMessage() method:

worker.postMessage(); // Start the worker.

For example, if you have requirement to prefetch some report based on user preference OR notify user when requested report is ready for retireval, you may script something like this on the main page,

...

<page onload="prepReport()"/>
<output id="result"/>

<script>
  function prepReport() {
    worker.postMessage({'cmd': 'submitReq', 'msg': reportParams});
  }
  var worker = new Worker('doRptWork.js');

  worker.addEventListener('message', function(e) {
    document.getElementById('result').textContent = e.data;
  }, false);
</script>

...

...

doRptWork.js:

self.addEventListener('message', function(e) {
  var data = e.data;
  switch (data.cmd) {
    case 'submitReq':
      self.postMessage('WORKER STARTED TO PROCESS REPORT REQ: ' + data.msg);

            .... Post an HTTP Request to the server ........

   break;    

     default: self.postMessage('Unknown command: ' + data.msg); };}, false);

Due to their multi-threaded behavior, web workers only has access to a subset of JavaScript's features:

• The navigator object
• The location object (read-only)
• XMLHttpRequest
• setTimeout()/clearTimeout() and setInterval()/clearInterval()
• The Application Cache
• Importing external scripts using the importScripts() method
• Spawning other web workers

Workers do NOT have access to:

• The DOM (it's not thread-safe)
• The window object
• The document object
• The parent object  

To check if your browser supports these features, visit http://www.findmebyip.com/ that will list what your browser does and does not support.

Shorter URI for Web - II

Back in June, under the post shorter-uris-for-business-apps.html I had illustrated how to shorten aesthetically unpleasant and difficult to remember long urls. Recently, I came across some emails from my alumni that had cryptic domain names and context paths (something like http://bit.ly/wX48sl) . Often encountering such shorter and spam kind of URL's all across the web world, I was curious to explore further.

It turned out these shorter versions are staged via url shortening service providers who registers and associates your longer version with some randomly generated alphanumeric token. When you distribute your urls, it will basically carry the service provider's domain along with this token; when users cilick on this shortened version, their browser is redirected to the longer version of the page. Personally, I do not like such URL's as they are often source of spams and phising; you need to trust the person sending the link as you cannot trust the link.

There are many players in this field like bit.ly, tr.im, cli.gs, etc., and such URL shortening is more popular with twitters. This link provides good comparision of features among all the players and feature set (http://searchengineland.com/analysis-which-url-shortening-service-should-you-use-17204)

HTML5 AppCache

Caching web pages has been around for quite a while, but its effectivness has always been a question. All browsers have caching mechanisms, but they're unreliable and don't always work as you might expect. Unless you explictly save the contents you may not be able to browse websites that you were online just a moment ago. HTML5 addresses some of the annoyances of being offline with the ApplicationCache interface a.k.a AppCache.

Using the cache interface gives your application three advantages:

1. Offline browsing - users can navigate your full site when they're offline. For business applications that require valid session, this scenrio could translate to caching static resources on user machines.
2. Speed - cached resources are local, and therefore load faster.
3. Reduced server load - the browser will only download resources from the server that have changed.

The AppCache allows a developer to specify which files the browser should cache and make available to offline users. Your app will load and work correctly, even if the user presses the refresh button while they're offline. To allow your existing application to take advantage of this feature you need to add "manifest" attribute to html tag and point it to appropriate cache manifest file which in turn controls which resources to be cached and which ones need network access.

The manifest attribute should be included on every page of your web application and can point to an absolute URL or relative path to your .manifest file. The file does not need to end in ".manifest", but needs to be served with the correct mime-type (see below).

A .manifest file must be served with the mime-type text/cache-manifest. You may need to add a custom file type to your web server or .htaccess configuration.

For example, to serve this mime-type in Apache, add this line to your config file:

AddType text/cache-manifest .manifest

The cache manifest file is a simple text file that lists the resources the browser should cache for offline access and may look something like this:

CACHE MANIFEST

index.html
stylesheet.css
images/logo.png
scripts/main.js

This example will cache four files on the site. The CACHE MANIFEST string is the first line and is required. Every URL or site is limited to 5MB worth of cached data.

A manifest can have three distinct sections: CACHE, NETWORK, and FALLBACK.

CACHE: 
    This is the default section for entries. Files listed under this header (or immediately after the CACHE MANIFEST) will be explicitly cached after they're downloaded for the first time.

NETWORK:

Files listed under this section are white-listed resources that require a connection to the server. All requests to these resources bypass the cache, even if the user is offline.

FALLBACK:

An optional section specifying fallback pages if a resource is inaccessible. The first URI is the resource, the second is the fallback. Both URIs must be relative and from the same origin as the .manifest file

A complex example like the one below can illustrate better on how these sections work,

CACHE MANIFEST

# Explicitly cached entries

CACHE:
index.html
stylesheet.css
images/logo.png
scripts/main.js

# Resources that require the user to be online.

NETWORK:
login.jsp
/mycontroller
http*://my.domain.com

# static.html will be served if main.py is inaccessible
# offline.jpg will be served in place of all images in images/large/

FALLBACK:

/main.htm /static.html
images/large/ images/offline.jpg

Resources configured under cache manifest can be updated/purged based on events fired by appcache API such as download progress, updating the app cache, and error conditions or programatically checking for life cycle status of the appcache object.

Cookie - 101

      Many developers understand cookies from a simple perspective that it is responsible for preserving state conversations in a web application and has an expiration interval. They also perhaps are aware that cookies can be created at the server side with a session identifier and pushed to client browser. Beyond this, many drew a blank when they are faced with issues such as timeouts, cross domain communication, and multi browser initiation.
  
      Under the hood there are two types, session and persistent cookies. The former has no lifetime beyond opening and closing of browser window, whereas the later can be retained for unlimited period of time and when created is written into browser cookie store on client machines. Besides, expiration intervals cookies have affinitiy to domains from which they originate. For example, "myspace.com", "client1.myspace.com", "secure.myspace.com", etc., can share the same cookie because they are either domains or subdomains of same type. This is easily achieved by explicitly setting "domain" property of cookie object to the other subdomain before streaming the output response (Most languages/scripts have cookie API with domain property). User apps can then navigate to and reference to links, images, scripts or resources within each other. However, if your infrastructure has systems in "myspace.com" and "myotherspace.com", then you will not be able to cross reference resources from each other. That would amount to cross site scripting attack and you may have to workaround at DNS level spoofing the IP addresses of these domains so that browsers can work without throwing "Internal Server Error".

     One other issue with cookies related to timeout emerge from modern browsers having tab feature. Users opening multiple browsers or tabs end up sharing the same cookie, meaning users are logged once but have multiple handles to the same server side session. This could pose serious problems to your applications depending on what they are trying to do. For example, user on one browser frame window may update a record and in parallel try to delete that record from other window or tab. Now that this user is attached to the same server session, there could be a data collision.

    If your application can allow multiple view access to same server session, for most part a convenience for users to compare and work across screens, then you may likely be limiting users to one session cookie and do not mind if they use multiple tab windows. However, if you want one and only one view representation of your system you just simply cannot rely on cookies. Cookies are controlled by window frame process and is shared across tab processes (in most browsers). On IE8, you have a feature to start a new session on your tab which decouples that tab from frame process thus establishing a new session with server. For browsers that do not have this feature, developers need to create home grown solutions like generating, encoding and attaching a windows handle ID to every URL request just like what one would do with sessionID if there were no cookies. On the server end you would associate this handle to one particular session token, user requests from other tabs or window processes will have different handle that will now be rejected. There are some server side frameworks (eg. Spring 3.2+) that can create cookies specific to tab handle instead of a javascript approach.

IBM WESB Tips and Aches - part II

IBM ESB v7.x has some nice improvements over its earlier one from both development and runtime standpoint. The earlier 6.x version lacked reusability and involved quite a bit of development for even the simplest tasks. With 7.x, IBM now supports XPATH 1.1 functions and additional primitives such as tracing, and mediation subflow which when properly designed can result in productive development.

First, to talk about tracing which is somewhat similar to logger, is quite handy at development time which simply streams your entire xml graph onto a specified/default file location. In production, you may enable/disable this feature by promoting and configuring boolean property. When you extend this feature across all modules you may want to have a generic reusable logger rather than one for each module. This is accomplished using mediation subflow that includes tracing primitive wired between its IN and OUT terminals.

Mediation subflows by design are intended to be reused across all modules and applications and therefore when working with those, care should be taken so that they are not created within "Application" or "Module" construct, but instead within a "Library" construct. This allows reuse across multiple modules or applications (ofcourse it also depends on what you have created within subflow).

One interesting note with mediation subflow, is accessing contextual information within the flow. Context information can either be correlation, transient or user context. Normally your business objects or custom BO's are bound to context and if you wanted to access those from within mediation subflow at design time, you may not find a handle to it. This is by design of mediation subflows which are supposed to be agnostic to your application or module and thus any type of contextual information. One way to get around this is to use "MessageSetter" prmitive as the first element within subflow and cast appropriate context (user, transient or correlation) with your business object. This facilitates visibility of context information within subflow for any wiring or transformation.

SOA operations verbage

Service Oriented capabilities like their object oriented counterpart follows standard naming conventions when it comes to naming the operations. Based on the message exchange pattern involved, these capabilities are either prefixed with one of the following list of verbs as used by the OAGIS:

• Acknowledge
• Cancel
• Change
• Confirm
• Get
• Sync
• Show
• Update
• Load
• Post
• Process
• Respond

For example, any one providing a financial calculator service may include capabilities like GetRate, ShowAnnualReturn, and UpdateAPY in their WSDL instead of some non-standard nomenclatures.