What is Reliability?

When architects or leads present one solution vs other one, they often speak of performance and throughput but quite ignore most of the other software attributes like maintainability, manageablity, availability, and reliability until the first failure. To accomplish each of these attributes in your sofware you need to follow certain design principles and build into your design rather hoping for it to be taken care at deployment time, though some of these are deployment attributes. In this post, I will discuss about the reliability factor which often is linked to availability and measured in 9's, for example a system is 99.9% available meaning the probability of system failure is 0.1 or 10%. So, how do one measure reliability. To understand, one need to agree on the definition of reliability.

There are actually two components that impact the reliability of a system

• How long it will work before it fails
• How long it will take to fix it.

We call the first the mean time before failure (MTBF) and the second the mean time to repair (MTR). In large transaction processing systems, reliability is often measured as down time. Down time has a cost associated with it – perhaps $1,000 per hour or $100,000 per hour. (Of course, MTR plays a role here as well, as the longer the down time, the higher the cost in many cases – from customer annoyance to lost sales to lost customers.) Down time alternatively can be measured as the proportion of time that a system is up, a measure that we call the availability of the system. Since the system is always either up or down, availability can be defined as,

A = MTBF/ MTBF + MTR

One can also define,

Availability = Doing the right thing = A

Failure = Doing the wrong thing = F = (1-A)

Reliability = Not doing the wrong thing = 1/(1-A)

To increase availability, the most intutive approach is to add redundant systems with full replication. However, depending on number of spares introduced in your topology, this introduces additional cost and replication issues. In general, more the number of nodes higher the chances of data collision, where users from two different systems try to update the same data. Another approach is to split the system so that under failure the other node assumes full load. Failure probability of this configuration is now,

F = n/k*(n/k-1)*(1-a)*(1-a)/2

where 'n' is number of elements or applications that run in that system and 'k' the number of nodes.

Rule of Thumb:

1. If a system is split into k parts, the resulting system network will be more than k times as reliable as the original system and still will deliver k/k-1 of the system capacity in the event of an outage.
2. If a system is split into k parts, the chance of losing more than 1/k of its capacity is many, many times less than the chance that the single system will lose all of its capacity.