Heap Dumps - Introduction

What Is a Heap Dump?

A heap dump is a snapshot of the memory of a Java process at a certain point of time. There are different formats for persisting this data, and depending on the format it may contain different pieces of information, but in general the snapshot contains information about the java objects and classes in the heap at the moment the snapshot was triggered. As it is just a snapshot at a given moment, a heap dump does not contain information such as when and where (in which method) an object was allocated.

What Are Heap Dumps Good for?

If there is a system which is crashing sporadically with an OutOfMemoryError, then analyzing an automatically written heap dump with MDD4J or IBM heap analyzer or MAT can be a very easy way to find the root cause of the problem. If you want to analyze what the footprint into memory of your application is, then MAT and heap dumps are again a good choice. This combination can also help you to find which are your biggest structures, to find redundant data structures, to find space wasted in unused collections, and much more.

If you however are trying to find out why too many garbage objects are produced during a certain operation, or want to see which methods allocate most of the objects, then you would need to use a profiler which is collecting data over time from the VM. Leak detecting techniques relying on analysis of the objects behavior (allocation / garbage collection) are difficult to implement using heap dumps (see object identity below).

Types of Heap Dumps

Currently the Memory Analyzer is able to work with HPROF binary heap dumps (produced by Sun, HP, SAP, etc… JVMs), IBM system dumps (after pre-processing them), and IBM portable heap dumps (PHD) from a variety of IBM platforms. Let’s have a closer look at each of the types.

HPROF Binary Heap Dumps

A detailed specification of the content of an HPROF file can be found here.

Below are summarized some of the important pieces of the information used within MAT:

Information about all loaded classes. For every class the HPROF dump contains its name, its super-class, its class loader, the defined fields for the instances (name and type), the static fields of the class and their values

Information about all objects. For every object one can find the class and the values of all fields – both references and primitive fields. The possibility to look at the names and the content of certain objects, e.g. the char[] within a huge StringBuilder, the size of a collection, etc … can be very helpful when performing memory analysis

a list of GC roots

the callstacks of all threads (in heap dumps from JDK 6 update 14 and above)

IBM System Dumps

On IBM platforms one can preprocess a system dump (core file) from a Java process with the jxtract tool, and analyze the result with Memory Analyzer on an arbitrary other box (DTFJ libraries have to be additionally installed, see details below in the “How to get heap dump” section). As the core file contains the whole process memory, this kind of dump also provides all the details seen in an hprof heap dump (including the field names, primitive fields’ values, stacktraces, etc…). There is even more information (e.g. process related information), but at the moment it is not used in Memory Analyzer.

IBM Portable Heap Dumps (PHD)

The PHD files are much smaller in size than the corresponding system dumps. However, they contain less information. The major difference between the HPROF dumps (or the IBM system dumps) and PHD dumps is that a PHD dump does not contain the values of the primitive fields. Only the non-null references from an object are provided. The second important difference is that the field names are not present, i.e. one can’t distinguish from which field a reference is made, and because of this the presented reference chains (paths) are not as concrete as with the other dumps. Using just the object graph is still enough for the analysis of many memory-related problems, but when the content of some fields is needed to get an idea why an object is too big then one has to use the system dumps.

Usually when a PHD dump is generated there is also a corresponding javacore file. If they are put together in the same directory when the PHD dump is opened with MAT, then some of the data in the javacore file will also be used.

So, having less information has both advantages and disadvantages - the PHD dumps are ways easier to transport from a customer (smaller size), can be used to find the biggest objects in the heap. And as they are usually written by default they are a good place to start the analysis. However, in some cases the information is enough to analyze in details the root cause of a problem.

How To Get a Heap DumpHow to obtain a heap dump depends on the platform and the used JVM. In general all VMs provide the possibility to request a heap dump manually, or to get one written from the VM when an OutOfMemoryError occurs. The second option is very convenient for the analysis of problems happening on production systems, or happening only sporadically, as one does not have to observe the system and wait for the problem to reoccur. A detailed description how a heap dump can be obtained depending on the JVM is provided here.

Are Dead Objects Present In the Heap Dump?Another question which often pops up is if garbage objects are included in the heap dump. This again depends on the heap dump, but usually a GC is done before the heap dump is written. Nevertheless there are always some objects which are unreachable from the GC roots, i.e. should be thrown away. The Memory Analyzer removes such objects during the initial parsing of a heap dump in order to simplify the analysis. If you want to have a look at the “garbage” or even want the objects to remain, then find here what to do.

Cloud Computing - 10 min Introduction

In my previous post on Virtualization I discussed various options adopted by vendors for virtualizing infrastructure and I/O ancillaries. Taking a step further, one can achieve cloud computing which is generally defined as "leveraging a pool of computing resources living within a datacenter with no sharing". The last two words "no sharing" is the key in cloud computing. Currently in most data centers, resources such as CPU, Memory, I/O are sliced and partitioned across applications based on anticipated capacity plans. However, these resources on nodes have a tight coupling to the applications and cannot be assigned to other applications without some serious deployment re-image. Think of Turbotax.com or other online retailers who have seasonal peak capacities for a month or two followed by very low volume for rest of the year. Capacity planning for these sites will involve building costlier hardware with utlization levels that will be low for most part of the year. There is simply no ability of scale up or down based on demand. The other business case could be your pre-production lower level environments that need to be juggled across many LOB's. Having a  cloud based infrastructure provides enormous advantage in these situations, where demand based scalability can be achieved at minimum cost. A cloud is composed of 5 essential characteristics, 3 service models and 4 deployment models. Key characteristics being on-demand self service, broad network access, resource pooling, rapid scaling, and measured service.

The two major service models defining cloud are, Infrastructure as a Service (IaaS) and Platform as a service (PaaS).
          IaaS involves building a distribution stack comprising of hardware and virutalized instance (the first two tiers), and rolling it out to users. Users are then free to install any OS on top of this stack with services, frameworks, and servers that can be tailored to support their needs. An example of this model is Amazon's EC2 public cloud. Under this model, user has freedom to choose anything from and above the operating system.
           PaaS involves a service model with its distribution stack composed of hardware, VMM and elements from all other tiers upto frameworks; user applications are deployed on top of vendor provided framework, an example is windows Azure. Users do not manage servers or storage and can only deploy apps on the cloud; above all their apps need to be coded to the framework built in PaaS.



These two models are quite different from another existing one, Software as a Service (SaaS), which is more of an utility computing where users can only use and not modify/install apps or any components in the underlying stack.

The 4 deployment models can be categorized into private, public, community and hybrid.

Private cloud. The cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on premise or off premise. (eg. your data center)

Community cloud. The cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be managed by the organizations or a third party and may exist on premise or off premise.

Public cloud. The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.

Hybrid cloud. The cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load-balancing between clouds).

The cloud computing industry represents a large ecosystem of many models, and vendors. These definitions attempt to encompass all of the various cloud approaches.

Virtualization - An Introduction

There is a lot of buzz going around in topics like SOA, Virtualization, Cloud Computing these days across all organizations, and if you haven't had any discussion with your senior management in exploring possibilities of adopting some of these in your organization then perhaps I should say you are not working for an IT enabled organization. In any case, I was stumped once when I was told software licensing cost may go down when systems are virtualized; not knowing why, I decided to dig under the hood and based on my findings here are some basic stuff on Virtualization.

Virtualization functions and external interfaces as their counterparts, but that differ in attributes, such as size, performance, and cost. These substitutes are called virtual resources, and their users are typically unaware of the substitution. Virtualization is commonly applied to physical hardware resources by combining multiple physical resources into shared pools from which users receive virtual resources. With virtualization, you can make one physical resource look like multiple virtual resources. Virtualization has been here for quite a while in different forms like, storage, VLAN, VIP, JVM, etc., but of late it has entered at the system level (OS).

System virtualization independent operating environments that use virtual resources. System virtualization can be approached through hardware partitioning or hypervisor technology. Hardware partitioning subdivides a physical server into fractions, each of which can run an operating system, our old dual boot systems. These fractions are typically created with coarse units of allocation, such as whole processors or physical boards. This type of virtualization allows for hardware consolidation, but does not have the full benefits of resource sharing and emulation offered by "hypervisors". To better understand hypervisors one need to understand the existing and most widely used x86 architecture below,

All these configurations have the ability to virutalize not just CPU's but also memory and I/O devices.

X86 operating systems are designed to run directly on the bare-metal hardware, so they naturally assume they fully ‘own’ the computer hardware. As shown in Figure, the x86 architecture offers four levels of privilege known as Ring 0, 1, 2 and 3 to operating systems and applications to manage access to the computer hardware. While user level applications typically run in Ring 3, the operating system needs to have direct access to the memory and hardware and must execute its privileged instructions in Ring 0. Virtualizing the x86 architecture requires placing a virtualization layer under the operating system (which expects to be in the most privileged Ring 0) to create and manage the virtual machines that deliver shared resources. This brings additional complexity while running/translating sensitive machine instructions which are specific to hardware vendor that cannot be generalized across. Three techniques were introduced to resolve this

• Virtualization using binary translation
• OS assisted virtualization or paravirtualization 
• Bare Metal virtualization
  

Virtualization via transalation is process intensive but involves no changes to OS or hardware, the hypervisor or virtual machine monitor (VMM) is installed on top of host computer OS as another application; to all user applications the hypervisor emulates as an operating system, consuming, and evaluating instructions and presenting results. The main advantage is OS sitting on top of VMM aka Guest OS can use CPU's from different host, the VMM abstracts these details.

OS assisted or paravirtualization involves installing OS that is aware of being virtualized and is mostly associated with open source like Linux. This is little faster than the previous one, but requires re-building your system with modified OS.

Bare Metal or Full Virtualization (below) involves the guest operating system running on top of a hypervisor that sits on the bare metal. The guest is unaware that it is being virtualized and assumes it is the only OS installed on the hardware. To install additional OS, VM Kernel would spawn additional VMM instance. This perhaps has the fastest performance of all other types.

All these techniques support not just CPU virutalization but also memory, device and I/O virtualization.

SOAD an introduction - I

There has been a lot of buzz and hype surrounding the opportunities presented by Service Oriented architecture (SOA) and its implementation as webservices. For many newbies SOA is all about service enabling components to some form of webservice and publishing them using standards. In reality, SOA is not about products and standards that help realize it, for example through webservice, but beyond those like services, their flows, compositions, quality of service, and governance. This brings us to service oriented modelling which is quite different from object oriented modelling in the sense that the later is more use case driven whereas the former is more process driven. In most cases, service oriented modelling is a hybrid of business process modelling, object oriented modelling and enterprise architecture.

To work on SOA projects, one may have to analyze, design and make architectural decisions at various layers like operational system or legacy layer, Enterprise component layer, Services layer, Business process or orchestration layer and the Integration layer.

Legacy layer : This consists of existing custom built applications, including existing CRM and ERP packaged applications, and older object-oriented system implementations, as well as business intelligence applications. The composite layered architecture of an SOA can leverage existing systems and integrate them using service-oriented integration techniques, which is a common case in most businesses.

Enterprise components layer: This is the layer of enterprise components that are responsible for realizing functionality and maintaining the QoS of the exposed services. This layer typically uses container-based technologies such as application servers to implement the components, workload management, high-availability, and load balancing. For example, components in these layers could be enterprise beans, asynch beans, message queues, thread pools, resource adapters, etc.,

Services layer: The services the business chooses to expose reside in this layer. They can be discovered or be statically bound and then invoked, or even, choreographed into a composite service. This service exposure layer also provides for the mechanism to take enterprise wide components, business unit specific components, and in some cases, project-specific components, and externalizes a subset of their interfaces in the form of service descriptions. Thus, the enterprise components provide service realization at runtime using the functionality provided by their interfaces. The interfaces get exported out as service descriptions in this layer, where they are exposed for use. They can exist in isolation or as a composite service.

Business process composition or choreography layer: Compositions and choreographies of services exposed in Service layer are defined in this layer. Services are bundled into a flow through orchestration or choreography, and thus act together as a single application. These applications support specific use cases and business processes.

Integration (ESB) Layer: This layer enables the integration of services through the introduction of a reliable set of capabilities, such as intelligent routing, protocol mediation, and other transformation mechanisms, often described as the ESB (the EDI equivalent). Web Services Description Language (WSDL) specifies a binding, which implies a location where the service is provided. On the other hand, an ESB provides a location independent mechanism for integration.

QoS Layer: This layer provides the capabilities required to monitor, manage, and maintain QoS such as performance, reliability, transaction, and availability. Some of these attributes are implemented at the Enterprise component layer (like transaction, security) whereas the others are monitored via background process that check the health of SOA applications.

              As it can be seen now, design strategy for a SOA does not start from bottom-up as is often the case with a Web services-based approach. SOA is more strategic and business-aligned, Web services are a tactical implementation of SOA. A number of important activities and decisions exist that influence not just integration architecture but enterprise and application architectures as well.

What is Reliability?

When architects or leads present one solution vs other one, they often speak of performance and throughput but quite ignore most of the other software attributes like maintainability, manageablity, availability, and reliability until the first failure. To accomplish each of these attributes in your sofware you need to follow certain design principles and build into your design rather hoping for it to be taken care at deployment time, though some of these are deployment attributes. In this post, I will discuss about the reliability factor which often is linked to availability and measured in 9's, for example a system is 99.9% available meaning the probability of system failure is 0.1 or 10%. So, how do one measure reliability. To understand, one need to agree on the definition of reliability.

There are actually two components that impact the reliability of a system

• How long it will work before it fails
• How long it will take to fix it.

We call the first the mean time before failure (MTBF) and the second the mean time to repair (MTR). In large transaction processing systems, reliability is often measured as down time. Down time has a cost associated with it – perhaps $1,000 per hour or $100,000 per hour. (Of course, MTR plays a role here as well, as the longer the down time, the higher the cost in many cases – from customer annoyance to lost sales to lost customers.) Down time alternatively can be measured as the proportion of time that a system is up, a measure that we call the availability of the system. Since the system is always either up or down, availability can be defined as,

A = MTBF/ MTBF + MTR

One can also define,

Availability = Doing the right thing = A

Failure = Doing the wrong thing = F = (1-A)

Reliability = Not doing the wrong thing = 1/(1-A)

To increase availability, the most intutive approach is to add redundant systems with full replication. However, depending on number of spares introduced in your topology, this introduces additional cost and replication issues. In general, more the number of nodes higher the chances of data collision, where users from two different systems try to update the same data. Another approach is to split the system so that under failure the other node assumes full load. Failure probability of this configuration is now,

F = n/k*(n/k-1)*(1-a)*(1-a)/2

where 'n' is number of elements or applications that run in that system and 'k' the number of nodes.

Rule of Thumb:

1. If a system is split into k parts, the resulting system network will be more than k times as reliable as the original system and still will deliver k/k-1 of the system capacity in the event of an outage.
2. If a system is split into k parts, the chance of losing more than 1/k of its capacity is many, many times less than the chance that the single system will lose all of its capacity.

JavaScript Threads

Most web developers must be aware (hopefully), that javascripts are executed on a single threaded environment within browser frame or tab process. At render time your scripts are executed sequentially and have to wait or yield until the previous request is complete. Many might have even noticed browser dialog

To mimic concurrency, developers use techniques like setTimeout(), setInterval(), XMLHttpRequest, and event handlers. All of these features run asynchronously, but non-blocking doesn't necessarily mean concurrency. Asynchronous events are processed after the current executing script has yielded. This is the primary reason why some page onload or javascript controlled behaviors for page rendering work most of the times but not always. Often these issues are blamed to slow network because you may not notice such behaviors every day.

The good news is HTML5 has put an end to this bringing threading into javascript world. Using Web workers, developers can now run heavy weight scripts in their own threads without blocking other components in the page. W3C specifications talks about two types of workers, shared and dedicated.

Dedicated Web Workers run in an isolated thread. As a result, the code that they execute needs to be contained in a separate file. But before we do that, the first thing to do is create a new Worker object in your main page. The constructor takes the name of the worker script:

var worker = new Worker('task.js');

If the specified file exists, the browser will spawn a new worker thread, which is downloaded asynchronously. The worker will not begin until the file has completely downloaded and executed. If the path to your worker returns an 404, the worker will fail silently. As with java threading, javascript threads do not return values or allow you to trap failures.

After instantiating the worker, you start it by calling the postMessage() method:

worker.postMessage(); // Start the worker.

For example, if you have requirement to prefetch some report based on user preference OR notify user when requested report is ready for retireval, you may script something like this on the main page,

...

<page onload="prepReport()"/>
<output id="result"/>

<script>
  function prepReport() {
    worker.postMessage({'cmd': 'submitReq', 'msg': reportParams});
  }
  var worker = new Worker('doRptWork.js');

  worker.addEventListener('message', function(e) {
    document.getElementById('result').textContent = e.data;
  }, false);
</script>

...

...

doRptWork.js:

self.addEventListener('message', function(e) {
  var data = e.data;
  switch (data.cmd) {
    case 'submitReq':
      self.postMessage('WORKER STARTED TO PROCESS REPORT REQ: ' + data.msg);

            .... Post an HTTP Request to the server ........

   break;    

     default: self.postMessage('Unknown command: ' + data.msg); };}, false);

Due to their multi-threaded behavior, web workers only has access to a subset of JavaScript's features:

• The navigator object
• The location object (read-only)
• XMLHttpRequest
• setTimeout()/clearTimeout() and setInterval()/clearInterval()
• The Application Cache
• Importing external scripts using the importScripts() method
• Spawning other web workers

Workers do NOT have access to:

• The DOM (it's not thread-safe)
• The window object
• The document object
• The parent object  

To check if your browser supports these features, visit http://www.findmebyip.com/ that will list what your browser does and does not support.

Shorter URI for Web - II

Back in June, under the post shorter-uris-for-business-apps.html I had illustrated how to shorten aesthetically unpleasant and difficult to remember long urls. Recently, I came across some emails from my alumni that had cryptic domain names and context paths (something like http://bit.ly/wX48sl) . Often encountering such shorter and spam kind of URL's all across the web world, I was curious to explore further.

It turned out these shorter versions are staged via url shortening service providers who registers and associates your longer version with some randomly generated alphanumeric token. When you distribute your urls, it will basically carry the service provider's domain along with this token; when users cilick on this shortened version, their browser is redirected to the longer version of the page. Personally, I do not like such URL's as they are often source of spams and phising; you need to trust the person sending the link as you cannot trust the link.

There are many players in this field like bit.ly, tr.im, cli.gs, etc., and such URL shortening is more popular with twitters. This link provides good comparision of features among all the players and feature set (http://searchengineland.com/analysis-which-url-shortening-service-should-you-use-17204)

HTML5 AppCache

Caching web pages has been around for quite a while, but its effectivness has always been a question. All browsers have caching mechanisms, but they're unreliable and don't always work as you might expect. Unless you explictly save the contents you may not be able to browse websites that you were online just a moment ago. HTML5 addresses some of the annoyances of being offline with the ApplicationCache interface a.k.a AppCache.

Using the cache interface gives your application three advantages:

1. Offline browsing - users can navigate your full site when they're offline. For business applications that require valid session, this scenrio could translate to caching static resources on user machines.
2. Speed - cached resources are local, and therefore load faster.
3. Reduced server load - the browser will only download resources from the server that have changed.

The AppCache allows a developer to specify which files the browser should cache and make available to offline users. Your app will load and work correctly, even if the user presses the refresh button while they're offline. To allow your existing application to take advantage of this feature you need to add "manifest" attribute to html tag and point it to appropriate cache manifest file which in turn controls which resources to be cached and which ones need network access.

The manifest attribute should be included on every page of your web application and can point to an absolute URL or relative path to your .manifest file. The file does not need to end in ".manifest", but needs to be served with the correct mime-type (see below).

A .manifest file must be served with the mime-type text/cache-manifest. You may need to add a custom file type to your web server or .htaccess configuration.

For example, to serve this mime-type in Apache, add this line to your config file:

AddType text/cache-manifest .manifest

The cache manifest file is a simple text file that lists the resources the browser should cache for offline access and may look something like this:

CACHE MANIFEST

index.html
stylesheet.css
images/logo.png
scripts/main.js

This example will cache four files on the site. The CACHE MANIFEST string is the first line and is required. Every URL or site is limited to 5MB worth of cached data.

A manifest can have three distinct sections: CACHE, NETWORK, and FALLBACK.

CACHE: 
    This is the default section for entries. Files listed under this header (or immediately after the CACHE MANIFEST) will be explicitly cached after they're downloaded for the first time.

NETWORK:

Files listed under this section are white-listed resources that require a connection to the server. All requests to these resources bypass the cache, even if the user is offline.

FALLBACK:

An optional section specifying fallback pages if a resource is inaccessible. The first URI is the resource, the second is the fallback. Both URIs must be relative and from the same origin as the .manifest file

A complex example like the one below can illustrate better on how these sections work,

CACHE MANIFEST

# Explicitly cached entries

CACHE:
index.html
stylesheet.css
images/logo.png
scripts/main.js

# Resources that require the user to be online.

NETWORK:
login.jsp
/mycontroller
http*://my.domain.com

# static.html will be served if main.py is inaccessible
# offline.jpg will be served in place of all images in images/large/

FALLBACK:

/main.htm /static.html
images/large/ images/offline.jpg

Resources configured under cache manifest can be updated/purged based on events fired by appcache API such as download progress, updating the app cache, and error conditions or programatically checking for life cycle status of the appcache object.

Cookie - 101

      Many developers understand cookies from a simple perspective that it is responsible for preserving state conversations in a web application and has an expiration interval. They also perhaps are aware that cookies can be created at the server side with a session identifier and pushed to client browser. Beyond this, many drew a blank when they are faced with issues such as timeouts, cross domain communication, and multi browser initiation.
  
      Under the hood there are two types, session and persistent cookies. The former has no lifetime beyond opening and closing of browser window, whereas the later can be retained for unlimited period of time and when created is written into browser cookie store on client machines. Besides, expiration intervals cookies have affinitiy to domains from which they originate. For example, "myspace.com", "client1.myspace.com", "secure.myspace.com", etc., can share the same cookie because they are either domains or subdomains of same type. This is easily achieved by explicitly setting "domain" property of cookie object to the other subdomain before streaming the output response (Most languages/scripts have cookie API with domain property). User apps can then navigate to and reference to links, images, scripts or resources within each other. However, if your infrastructure has systems in "myspace.com" and "myotherspace.com", then you will not be able to cross reference resources from each other. That would amount to cross site scripting attack and you may have to workaround at DNS level spoofing the IP addresses of these domains so that browsers can work without throwing "Internal Server Error".

     One other issue with cookies related to timeout emerge from modern browsers having tab feature. Users opening multiple browsers or tabs end up sharing the same cookie, meaning users are logged once but have multiple handles to the same server side session. This could pose serious problems to your applications depending on what they are trying to do. For example, user on one browser frame window may update a record and in parallel try to delete that record from other window or tab. Now that this user is attached to the same server session, there could be a data collision.

    If your application can allow multiple view access to same server session, for most part a convenience for users to compare and work across screens, then you may likely be limiting users to one session cookie and do not mind if they use multiple tab windows. However, if you want one and only one view representation of your system you just simply cannot rely on cookies. Cookies are controlled by window frame process and is shared across tab processes (in most browsers). On IE8, you have a feature to start a new session on your tab which decouples that tab from frame process thus establishing a new session with server. For browsers that do not have this feature, developers need to create home grown solutions like generating, encoding and attaching a windows handle ID to every URL request just like what one would do with sessionID if there were no cookies. On the server end you would associate this handle to one particular session token, user requests from other tabs or window processes will have different handle that will now be rejected. There are some server side frameworks (eg. Spring 3.2+) that can create cookies specific to tab handle instead of a javascript approach.

IBM WESB Tips and Aches - part II

IBM ESB v7.x has some nice improvements over its earlier one from both development and runtime standpoint. The earlier 6.x version lacked reusability and involved quite a bit of development for even the simplest tasks. With 7.x, IBM now supports XPATH 1.1 functions and additional primitives such as tracing, and mediation subflow which when properly designed can result in productive development.

First, to talk about tracing which is somewhat similar to logger, is quite handy at development time which simply streams your entire xml graph onto a specified/default file location. In production, you may enable/disable this feature by promoting and configuring boolean property. When you extend this feature across all modules you may want to have a generic reusable logger rather than one for each module. This is accomplished using mediation subflow that includes tracing primitive wired between its IN and OUT terminals.

Mediation subflows by design are intended to be reused across all modules and applications and therefore when working with those, care should be taken so that they are not created within "Application" or "Module" construct, but instead within a "Library" construct. This allows reuse across multiple modules or applications (ofcourse it also depends on what you have created within subflow).

One interesting note with mediation subflow, is accessing contextual information within the flow. Context information can either be correlation, transient or user context. Normally your business objects or custom BO's are bound to context and if you wanted to access those from within mediation subflow at design time, you may not find a handle to it. This is by design of mediation subflows which are supposed to be agnostic to your application or module and thus any type of contextual information. One way to get around this is to use "MessageSetter" prmitive as the first element within subflow and cast appropriate context (user, transient or correlation) with your business object. This facilitates visibility of context information within subflow for any wiring or transformation.

SOA operations verbage

Service Oriented capabilities like their object oriented counterpart follows standard naming conventions when it comes to naming the operations. Based on the message exchange pattern involved, these capabilities are either prefixed with one of the following list of verbs as used by the OAGIS:

• Acknowledge
• Cancel
• Change
• Confirm
• Get
• Sync
• Show
• Update
• Load
• Post
• Process
• Respond

For example, any one providing a financial calculator service may include capabilities like GetRate, ShowAnnualReturn, and UpdateAPY in their WSDL instead of some non-standard nomenclatures.

Password security

Securing passwords and other sensitive information is not new to anyone. But the approach of how to secure and handle those information in your application seem to differ across many developers. I have seen in some places, passwords being Base64 encoded so that it is convenient for QA or support folks to troubleshoot or provide back door support to their clients. In some other places, they were doing 3DES encryption but within application they were decrypted to plain text. Even worse, a minor section of folks had it in clear text all over.

Security is a cross cutting concern, meaning these sensitive packets need to be encrypted the moment they leave user view space and maintained in the same form during transmission, reception, processing, and storage. It should not be decrypted at any stage, not even by your application (since those may end up in some log file).

The most common form is MD5 one way hashing where you cannot recover the original message. However, developers need to keep in mind this type of encryption has dependency on OS address space, i.e 32-bit or 64-bit and applied character encoding. You may not be able to produce a matching MD5 output when one of your app produces MD5 on 32-bit and another app validates MD5 of the same text using 64-bit.

Who is an Architect

This may not be a new discussion for many, but I am writing this out of recent observations when people of all walks of IT think themselves doing some kind of architectural work. Every computer programmer, system designer, IT support director, computer operator, PC owner, data architect, database architect, network architect, business analyst, systems analyst, enterprise architect, service architect, object architect, project manager calls whatever they do or want to do or maybe, whatever, they are doing as, “Architecture?" It is chaos. As John Zachman puts it, "This is killing Enterprise Architecture…". So what is Enterprise architecture from an IT perspective?

          Enterprise Architecture when defined from an information and technology perspective explicitly describes an organization through a set of independent, non-redundant artifacts, defining how these artifacts interrelate with each other, and developing a set of prioritized, aligned initiatives and road maps to understand the organization, communicate this understanding to stakeholders, and move the organization forward to its desired state.

        From a Business perspective, Enterprise Architecture illuminates how an organization and all of its members can achieve its objectives, through the creation of a series of engineered models and project initiatives, that can be easily understood by all of the people associated with the organization.

       If an individuals contribution is to just about talk on these artifacts (or even worse BS on those) and not work towards creating them, please stop calling yourself an Enterprise Architect. May be they can be called as "Art Vandelay" Architect as in one of SeinField's episode.

Websphere JNDI resource scoping

Most folks have their application deployed at cluster level and so the resources attached to them such as Database, JMS, etc., I was challenged by my websphere administrator the need to create database resources at cluster level when applications created at different clusters talk to same database. My initial hunch was these resources are isolated under cluster and therefore will not have resource contention, however when they are moved to cell level their isolation is lost and resources will now be shared by all application clusters.

Interestingly this wasn't true. Even though IBM marks resources created at cell level with broader scope, when it comes to connections each node gets its own share. Despite the scope of a defined resource, the resource properties apply at an individual server level only.The application scope has precedence over all the scopes.

• The server scope has precedence over the node and cell scopes.
• The node scope has precedence over the cell scope.

For example, if you define the scope of a data source at the cell level, all the users in that cell can look up and use that data source, which is unique within that cell. However, resource property settings are local to each server in the cell. For example, if you define the maximum connections as 10, then each server in that cell can have 10 connections.

The cell scope is the most general scope and does not override any other scope. The recommendation is that you generally specify a more specific scope than the cell scope. When you define a resource at a more specific scope, you provide greater isolation for the resource. When you define a resource at a more general scope, you provide less isolation. Greater exposure to cross-application conflicts occur for a resource that you define at a more general scope.

MDB controller

Message driven beans and other resource adapters based on JCA technology have lifecycle stages that can be controlled for precise timing of when a message is to be read from queue or when a file need to be read from a location. Prior to EJB 2.1, MDB's were just message listeners based on listener ports and could read messages only from JMS queues. With subsequent revisions they were based on JCA connector architecture which provided flexibility to have MDB's to not just listen JMS queues but also file systems, ofcourse proper listener need to be implemented. In addition, JMX extensions on these components provide non-intrusive way to control the lifecycle of JCA, which is Initialize, passivate and activate.
I came across a requirement to fetch messages dropped into the queue only at defined intervals and not just when message was available. Websphere JMX AdminService API exposes provides operations that can interrogate available J2CMessage bindings and issue commands to either pause, resume the endpoint listeners. One caveat though is these activation specs need to be initialized within your application for AdminService to look them up. In other words, if you have multiple EAR's encapsulated with MDB's and if you wish to control them centrally through another EAR (all deployed on same cell), you will not be able to passivate without them being activated first.

String endPointlistener = "*:*,type=J2CMessageEndpoint";

javax.management.ObjectName queryName =
new javax.management.ObjectName( endPointlistener);

AdminService adminService = AdminServiceFactory.getAdminService();
Set s = adminService.queryNames(queryName, null);
Iterator it = s.iterator();
List mbeans = new ArrayList();
while (it.hasNext()) {
ObjectName mBean = (ObjectName) it.next();
result = adminService.invoke(mBean, operationName, params,
signature);
}

Dynamic JCA Connection Factory Substitution

JCA connection factories can be categorized logically into two sections. One that manages connection details and other interaction details. In websphere (and I presume the same with other application servers) you have to configure connection details on these factories as part administrative action. I had a requirement where connection host was fixed but authentication information for the host was different based on scenarios. Unless there was a way to dynamically substitute security information you would end up creating multiple connection factories.

In websphere you can set custom properties using the prefix "CP" to your resource property within your application. Ofcourse this will involve using IBM specific API within your application code to lookup connection factory and dynamically substituting connection details. Most of JCA Connection properties can be set using this approach except the hostname.

For example, to you may have a Connection factory marked against a file resource adapter, with JNDI com/eis/myFileCF dedicated for I/O write operations. Supposing, there are many folders that are operated by certain users who have privilige to delete while others have only write access it may become a daunting task to create dedicated connection factories to isolate these set of users. Instead application developers can lookup the connection factory by JNDI and inject any property variances such as username and password, in this case, like
                 conn.setProperty("CPuserName", "user");
                 conn.setProperty("CPpassword", "{xor}49dksdldk");

   Note, the password is masked using xor alogrithm. Websphere encodes J2C credentials while saving into file system and later decodes when sending them out to appropriate endpoints; your dynamic injection should encode the same way for proper processing.

Shorter URI's for web apps

Some non serious IT service cos develop web apps and roll it out to customers without paying attention to little details like URI's, UI component names they choose in HTTP communication, information they drop in cookies, etc., They little realize these oversights pose potential security risk to their application, especially when they are exposed over internet not to mention how silly it looks to expose your class name or object responsibility to users of the system. Some common examples look like,

http://my.domain.com/myapp/RegistrationServlet?userid=abc&password=bbb&address=blah

Issue 1: You indicate what technology you implement and which object is working your request; the complex your app the longer the url normally tend to be.
Issue 2: Whether you do FORM POST or pass it in as URL args, attaching element names like userid, password, etc., provide obvious clues to what you intend to do. Not a good practice.

With such apps the common dilemma we all face is not wanting to invest time working on nomenclatures. Luckily, webservers come to our rescue

Based on your deployment architecture, whether you have an application server + webserver or just webserver, one can use either mod_rewrite or mod_proxy modules of apache that can shield longer URL's from user with a short one.

If your app lives only on webserver, say a php or asp based application you can rewrite the above example like,

RewriteEngine on
RewriteRule ^/reg$ /myapp/RegistrationServlet
RewriteCond %{QUERY_STRING} id=([-a-zA-Z0-9_+]+)
RewriteRule ^/?reg$ %1? [R=301,L]

which is basically same as http://my.domain.com/reg/abc/abb/blah

If your app has webserver fronting an application server you cannot just rewrite, you may have to forward proxy short url requests to long one internally before some kind of plugin can forward that request to application server.


SQL Gotchas

While skimming my old zip drives I came across this interesting note that I had written years back related to SQL statements and prepared statements using iBatis framework. It was something to do with datatype of columns and implicit joins using prepared statement.

Problem: Issue of a static sql statement returns expected results, same statement issued as a prepared statment with parameters fails.

Static statement:

SELECT
nvl(sum(BATCHREL), 0) as CNT
FROM
userapplications a,
users b,
customers c
WHERE
a.USERRID = b.RID and
b.CUSTOMERKEY = c.RID AND
lower(c.ID) = '900100ev' AND
lower(b.ID) = 'admin1'

Prepared statment template:

SELECT
nvl(sum(BATCHREL), 0) as CNT
FROM
userapplications a,
users b,
customers c
WHERE
a.USERRID = b.RID and
b.CUSTOMERKEY = c.RID AND
lower(c.ID) = ? AND
lower(b.ID) = ?

Parameter values: '900100ev','admin1'



What I saw happening:

The c.id metadata was CHAR(20) and did not match the parameter passed in using prepared stmt, unless I padded it with spaces (to meet the length), however when I chose a literal string it worked w/out issues. I had to modify the prepared statement as follows (and it works)

SELECT
nvl(sum(BATCHREL), 0) as CNT
FROM
userapplications a,
users b,
customers c
WHERE
a.USERRID = b.RID and
b.CUSTOMERKEY = c.RID AND
RTRIM(lower(c.ID)) = #companyid# AND
lower(b.ID) = #operatorid#

Parameter values: '900100ev','admin1'

Mobile Web Applications

Recently a colleague of mine was discussing developing mobile apps for smartphones when I recollected my past engagement where I had to architect a digital coupon system running on web and mobile platforms with single code base. It presented quite a challenge given that your web apps are content rich and mobile devices had limited real estate and each device had different ergonomics.

Here are some lessons learnt from my experience -

Before you dive on mobile enabling consumer, read these design constraints and please come up design notes/solutions that will address each one of them.

Web and mobile are different media serving different requirements and different fields of application. Web mobile will be targetted towards 3 segments: Feature phones, smart phones, PDA's

• Feature phones usually come in candy bar, clamshell or slider form. They have a 12-key layout.
• Smart phones share the same features as a feature phone with two primary differences: Its ability to run additional third-party applications and a slightly larger screen. Smart phones typically use a more full featured operating system.
• PDAs often include QWERTY keyboard and stylus in place of the 12-key layout on normal phones.They also feature a larger screen.
  

Design constraints:

• Mobile browsers may stop scrolling vertically on pressing the down key when a link appears in the currently viewed section of the page.
• Some user plans may be tied to data bandwidth and therefore keep the data traffic to a minimum.
• Mobile devices have limited memory, limited CPU and run on slower networks than desktop computers, running scripts on client side will drain the battery. Keep such stuff to a minimum.
• Mobiles do not support popups/frames/etc. Not all mobiles support scripting/styling. Identify mobile device and then render pages as per device support features. Page Design will need to take account for diff. screen sizes, devices.
• The W3C suggest that all the images on a page should total less than 10KB. Keep page size to less than 25kb.
• You should consider how your page will render when the images are not available
  

Rules to Follow:

• Use HTML Validator firefox extn to validate markups for every page.
• Use style based layout instead of tables to produce pages. Page layout will be split as header, content. while you should state the dimensions of your images (width="120" height="107") when designing for mobile devices, don't state the pixel units. Keep dimensions dynamic that will be supplied from, say a DeviceBean.
• Use accesskeys to links and limit links to less than 10. Navigation can be performed using keys instead of browsing.
• Use Wireless CSS
• Adopt a short URL
• Do not mask user input when entering a password.
• Read http://www.w3.org/TR/mobile-bp/
  

IBM WESB tips and aches - I

SOA support in IBM has improved a lot in their current release WESB7.x compared with earlier one, however it still is mired with issues that cause frustration. We were at v6.x struggling to figure out why's and where's of our friendly WID tool, a resource gobbler that would bring your development machine to its knees, when sudden developments beyond our control forced us to move to v7.x. The oddity being our Dev, staging and prod environments were to be rebuilt rather than running an upgrade raising eyebrows and suspicions.

Instead of building from Dev-> QA ->Staging->Prod in that order, our learned friends chose the reverse order so that development was not impacted. They were assuming repeatable and reliable behavior of topology images from a tool they had no prior experience. Alas, surprise in store, all went well in Prod but when executing the same instructions in Staging we noticed weird issues pointing to SpringFramework and J2C transactions. The surprise initially was not the failure issues but the springframework in IBM, our corporate folks always shunned from open source tools and were a big fan of IBM, EJB, et al and vehemently opposed introducing Spring. Now here it is backed by the big blue spewing errors that no one can make sense.

After couple of days of research the issue was nailed to datasources "jdbc/mediationMessageLog" and "jdbc/WPSDB", where both share connections and enlist in XA transaction. For some reason enlistment was failing even though XA was enabled at datasource and backend. Once we changed datasource property to use non-sharable and non transactional connection the problem went away. Ofcourse, as a side effect we were not able to use messagelog primitive which uses jdbc/mediationMessageLog for database logging. No big deal, we used trace log as a workaround to meet deadlines while waiting on IBM to figure out this anomaly.

Java 5 Thread framework

To develop solutions related to work load management or batch processing or job parallelism, developers invented their own design using standard classes from java thread package or utilities provided by J2EE container based mechanisms like commonj framework. Implementing homegrown solutions using Thread or Runnable interfaces had serious limitations that it cannot communicate back the status of the job executed, or cannot be reused across many job requests. Besides, the low level concurrency primitives such as synchronized, notify, wait, etc., are either difficult to use correctly or difficult to debug and detect in case of errors.

Recently, I encountered one such implementation developed during Java 2.0 days. It was homegrown framework designed to handle critical batch jobs of high volume. It seem to have worked well for business which surprised me and after a little bit of probing, it was evident the application support would intervene and restart the job processes if there were failures, and besides batches were not bound by any SLA. To speed up execution cycle developers were spawning threads in a loop controlled by sleep intervals. Not a design one would want if it executes critical business processes.
With Java 5.0+, a lot has changed in the threading space with the inclusion of concurrent libraries (earlier these were standalone third party binaries without concurrent collections and scheduling framework). Concurrency utilities provide many advantages to developer community like,

# Reduced programming effort. It is far easier to use a standard class than to develop it yourself.
# Increased performance. The implementations in the Concurrency Utilities were developed and peer-reviewed by concurrency and performance experts; these implementations are likely to be faster and more scalable than a typical implementation, even by a skilled developer.
# Increased reliability. By using standardized, extensively tested concurrency building blocks, many potential sources of threading hazards such as deadlock, starvation, race conditions, or excessive context switching are eliminated.
# Improved maintainability. Programs which use standard library classes are easier to understand and maintain than those which rely on complicated, homegrown classes.
# Increased productivity. Now that it is part of Java API, developers are more likely to be familiar than any third party libraries and behavior.