Heap Dumps - Introduction

What Is a Heap Dump?

A heap dump is a snapshot of the memory of a Java process at a certain point of time. There are different formats for persisting this data, and depending on the format it may contain different pieces of information, but in general the snapshot contains information about the java objects and classes in the heap at the moment the snapshot was triggered. As it is just a snapshot at a given moment, a heap dump does not contain information such as when and where (in which method) an object was allocated.

What Are Heap Dumps Good for?

If there is a system which is crashing sporadically with an OutOfMemoryError, then analyzing an automatically written heap dump with MDD4J or IBM heap analyzer or MAT can be a very easy way to find the root cause of the problem. If you want to analyze what the footprint into memory of your application is, then MAT and heap dumps are again a good choice. This combination can also help you to find which are your biggest structures, to find redundant data structures, to find space wasted in unused collections, and much more.

If you however are trying to find out why too many garbage objects are produced during a certain operation, or want to see which methods allocate most of the objects, then you would need to use a profiler which is collecting data over time from the VM. Leak detecting techniques relying on analysis of the objects behavior (allocation / garbage collection) are difficult to implement using heap dumps (see object identity below).

Types of Heap Dumps

Currently the Memory Analyzer is able to work with HPROF binary heap dumps (produced by Sun, HP, SAP, etc… JVMs), IBM system dumps (after pre-processing them), and IBM portable heap dumps (PHD) from a variety of IBM platforms. Let’s have a closer look at each of the types.

HPROF Binary Heap Dumps

A detailed specification of the content of an HPROF file can be found here.

Below are summarized some of the important pieces of the information used within MAT:

Information about all loaded classes. For every class the HPROF dump contains its name, its super-class, its class loader, the defined fields for the instances (name and type), the static fields of the class and their values

Information about all objects. For every object one can find the class and the values of all fields – both references and primitive fields. The possibility to look at the names and the content of certain objects, e.g. the char[] within a huge StringBuilder, the size of a collection, etc … can be very helpful when performing memory analysis

a list of GC roots

the callstacks of all threads (in heap dumps from JDK 6 update 14 and above)

IBM System Dumps

On IBM platforms one can preprocess a system dump (core file) from a Java process with the jxtract tool, and analyze the result with Memory Analyzer on an arbitrary other box (DTFJ libraries have to be additionally installed, see details below in the “How to get heap dump” section). As the core file contains the whole process memory, this kind of dump also provides all the details seen in an hprof heap dump (including the field names, primitive fields’ values, stacktraces, etc…). There is even more information (e.g. process related information), but at the moment it is not used in Memory Analyzer.

IBM Portable Heap Dumps (PHD)

The PHD files are much smaller in size than the corresponding system dumps. However, they contain less information. The major difference between the HPROF dumps (or the IBM system dumps) and PHD dumps is that a PHD dump does not contain the values of the primitive fields. Only the non-null references from an object are provided. The second important difference is that the field names are not present, i.e. one can’t distinguish from which field a reference is made, and because of this the presented reference chains (paths) are not as concrete as with the other dumps. Using just the object graph is still enough for the analysis of many memory-related problems, but when the content of some fields is needed to get an idea why an object is too big then one has to use the system dumps.

Usually when a PHD dump is generated there is also a corresponding javacore file. If they are put together in the same directory when the PHD dump is opened with MAT, then some of the data in the javacore file will also be used.

So, having less information has both advantages and disadvantages - the PHD dumps are ways easier to transport from a customer (smaller size), can be used to find the biggest objects in the heap. And as they are usually written by default they are a good place to start the analysis. However, in some cases the information is enough to analyze in details the root cause of a problem.

How To Get a Heap DumpHow to obtain a heap dump depends on the platform and the used JVM. In general all VMs provide the possibility to request a heap dump manually, or to get one written from the VM when an OutOfMemoryError occurs. The second option is very convenient for the analysis of problems happening on production systems, or happening only sporadically, as one does not have to observe the system and wait for the problem to reoccur. A detailed description how a heap dump can be obtained depending on the JVM is provided here.

Are Dead Objects Present In the Heap Dump?Another question which often pops up is if garbage objects are included in the heap dump. This again depends on the heap dump, but usually a GC is done before the heap dump is written. Nevertheless there are always some objects which are unreachable from the GC roots, i.e. should be thrown away. The Memory Analyzer removes such objects during the initial parsing of a heap dump in order to simplify the analysis. If you want to have a look at the “garbage” or even want the objects to remain, then find here what to do.

Cloud Computing - 10 min Introduction

In my previous post on Virtualization I discussed various options adopted by vendors for virtualizing infrastructure and I/O ancillaries. Taking a step further, one can achieve cloud computing which is generally defined as "leveraging a pool of computing resources living within a datacenter with no sharing". The last two words "no sharing" is the key in cloud computing. Currently in most data centers, resources such as CPU, Memory, I/O are sliced and partitioned across applications based on anticipated capacity plans. However, these resources on nodes have a tight coupling to the applications and cannot be assigned to other applications without some serious deployment re-image. Think of Turbotax.com or other online retailers who have seasonal peak capacities for a month or two followed by very low volume for rest of the year. Capacity planning for these sites will involve building costlier hardware with utlization levels that will be low for most part of the year. There is simply no ability of scale up or down based on demand. The other business case could be your pre-production lower level environments that need to be juggled across many LOB's. Having a  cloud based infrastructure provides enormous advantage in these situations, where demand based scalability can be achieved at minimum cost. A cloud is composed of 5 essential characteristics, 3 service models and 4 deployment models. Key characteristics being on-demand self service, broad network access, resource pooling, rapid scaling, and measured service.

The two major service models defining cloud are, Infrastructure as a Service (IaaS) and Platform as a service (PaaS).
          IaaS involves building a distribution stack comprising of hardware and virutalized instance (the first two tiers), and rolling it out to users. Users are then free to install any OS on top of this stack with services, frameworks, and servers that can be tailored to support their needs. An example of this model is Amazon's EC2 public cloud. Under this model, user has freedom to choose anything from and above the operating system.
           PaaS involves a service model with its distribution stack composed of hardware, VMM and elements from all other tiers upto frameworks; user applications are deployed on top of vendor provided framework, an example is windows Azure. Users do not manage servers or storage and can only deploy apps on the cloud; above all their apps need to be coded to the framework built in PaaS.



These two models are quite different from another existing one, Software as a Service (SaaS), which is more of an utility computing where users can only use and not modify/install apps or any components in the underlying stack.

The 4 deployment models can be categorized into private, public, community and hybrid.

Private cloud. The cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on premise or off premise. (eg. your data center)

Community cloud. The cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be managed by the organizations or a third party and may exist on premise or off premise.

Public cloud. The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.

Hybrid cloud. The cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load-balancing between clouds).

The cloud computing industry represents a large ecosystem of many models, and vendors. These definitions attempt to encompass all of the various cloud approaches.

Virtualization - An Introduction

There is a lot of buzz going around in topics like SOA, Virtualization, Cloud Computing these days across all organizations, and if you haven't had any discussion with your senior management in exploring possibilities of adopting some of these in your organization then perhaps I should say you are not working for an IT enabled organization. In any case, I was stumped once when I was told software licensing cost may go down when systems are virtualized; not knowing why, I decided to dig under the hood and based on my findings here are some basic stuff on Virtualization.

Virtualization functions and external interfaces as their counterparts, but that differ in attributes, such as size, performance, and cost. These substitutes are called virtual resources, and their users are typically unaware of the substitution. Virtualization is commonly applied to physical hardware resources by combining multiple physical resources into shared pools from which users receive virtual resources. With virtualization, you can make one physical resource look like multiple virtual resources. Virtualization has been here for quite a while in different forms like, storage, VLAN, VIP, JVM, etc., but of late it has entered at the system level (OS).

System virtualization independent operating environments that use virtual resources. System virtualization can be approached through hardware partitioning or hypervisor technology. Hardware partitioning subdivides a physical server into fractions, each of which can run an operating system, our old dual boot systems. These fractions are typically created with coarse units of allocation, such as whole processors or physical boards. This type of virtualization allows for hardware consolidation, but does not have the full benefits of resource sharing and emulation offered by "hypervisors". To better understand hypervisors one need to understand the existing and most widely used x86 architecture below,

All these configurations have the ability to virutalize not just CPU's but also memory and I/O devices.

X86 operating systems are designed to run directly on the bare-metal hardware, so they naturally assume they fully ‘own’ the computer hardware. As shown in Figure, the x86 architecture offers four levels of privilege known as Ring 0, 1, 2 and 3 to operating systems and applications to manage access to the computer hardware. While user level applications typically run in Ring 3, the operating system needs to have direct access to the memory and hardware and must execute its privileged instructions in Ring 0. Virtualizing the x86 architecture requires placing a virtualization layer under the operating system (which expects to be in the most privileged Ring 0) to create and manage the virtual machines that deliver shared resources. This brings additional complexity while running/translating sensitive machine instructions which are specific to hardware vendor that cannot be generalized across. Three techniques were introduced to resolve this

• Virtualization using binary translation
• OS assisted virtualization or paravirtualization 
• Bare Metal virtualization
  

Virtualization via transalation is process intensive but involves no changes to OS or hardware, the hypervisor or virtual machine monitor (VMM) is installed on top of host computer OS as another application; to all user applications the hypervisor emulates as an operating system, consuming, and evaluating instructions and presenting results. The main advantage is OS sitting on top of VMM aka Guest OS can use CPU's from different host, the VMM abstracts these details.

OS assisted or paravirtualization involves installing OS that is aware of being virtualized and is mostly associated with open source like Linux. This is little faster than the previous one, but requires re-building your system with modified OS.

Bare Metal or Full Virtualization (below) involves the guest operating system running on top of a hypervisor that sits on the bare metal. The guest is unaware that it is being virtualized and assumes it is the only OS installed on the hardware. To install additional OS, VM Kernel would spawn additional VMM instance. This perhaps has the fastest performance of all other types.

All these techniques support not just CPU virutalization but also memory, device and I/O virtualization.