The BCP Blog has Moved to www.mha-it.com/blog

All,

The BCP Blog has a new home.  The BCP blog is now part of the MHA Consulting website at http://mha-it.com/blog/

Thank you.

Advertisements
Posted in Uncategorized. Leave a Comment »

Assessing True IT DR Recovery Capability with Tier 2 Metrics

Metrics are key to assessing the maturity and level of sophistication of your continuity planning program.  We believe there are two (2) levels of metrics you can use to assess your program.  Tier 1 metrics address the underpinnings of the program while Tier 2 metrics address what is the real recovery capability.  Too many times, only Tier 1 metrics are used and give a false sense of recoverability to senior management.  Tier 2 metrics, if used properly, will paint the real picture of the DR capability of the organization.  Here are examples of Tier 1 and 2 metrics:

Example:  A large organization has had a DR office and program in place for many years, multiple backup sites in place and holds recovery exercises on a regular basis throughout the year.  Internal audit regularly gives them their seal of approval.  Sounds good, right?  On the surface, the Tier 1 metrics will paint an average or above average picture of the program and its adherence to best practices.  Now, once we used Tier 2 metrics to assess the real recovery capability of the organization, it was quickly determined that the true IT disaster recovery capability did not exist and that the DR program was failing miserably in its ability to recover the critical systems and applications of the organization. Senior management in IT had lived under the belief that all was well in the realm of DR when in reality, could not even begin to recover the systems and applications needed to run the business.

Our metrics tool analyzes the level of compliance, weights them and produces speedometer graphs that quickly show management the state of the component.  Additionally, our tool ensures that if key components in an area are below average, the entire component cannot be raised even if its score dictates it.  Using metrics is painful and can be eye-opening but also can lead to increased management support, allocation of resources and most importantly, the multi-year budget needed to implement what you need to succeed.  Use them, they work.

Crisis Management – Manage by Title or Capability?

Do you manage your Crisis Management teams by title or capability?  In previous posts, I have discussed a common issue that many Crisis Management Teams suffer across different industries and that is having people lead or serve on their teams simply due to their title or position within the organization.

We must change our mindset and manage our Crisis Management Team by those with the capability and authority to strategically respond and recover from an unplanned event.  A client of ours experienced a highly publicized event regarding their CEO that required their Incident Management team to be assembled for a long period of time and quickly found that  a number of team members placed on the team by title, needed to be replaced by others who could execute.  The members were released and the team members who could execute, continued their efforts.  The event forever changed how this client will manage their Crisis Management Team.  Managing your department well on a day to day basis and being able to manage the team through a crisis are two totally different events.

We must look at our teams like a professional sports team and identify the strengths and weaknesses from the Leader to the individual role players.  In my consulting firm, we rank our consultants by capability in client interaction,  ability to lead engagements, work behind the scenes, etc.  By doing this, we are able to place the best consultant fit for the particular engagement.

This is not an easy task to complete as Crisis Management Teams are typically filled with politics and alliances across the senior management team who typically fills these roles.  To assess team performance, we recommend that you have primary and alternate team members participate in mock disaster exercises, have different team leaders run the exercises and stress test individuals on the team by requiring specific tasks to be completed.  By doing this, you can assess team member performance, strength and weaknesses.

On a regular basis, sit down with your senior management and ask them to rank the Crisis Management team and its players in its ability to execute.  Discuss the strengths and opportunities of the team and remind senior management that its the ability to execute in a crisis is what a team member must be able to do.  Lets look beyond the title and bring in the people who can lead the organization.

Are IT DR Budget Comparisons Always Indicative of a Good Spend?

We are often asked by our clients to compare their Disaster Recovery spend and staffing to like organizations and industries as part of our Disaster Recovery Assessment engagements.  This spend comparison typically includes number of staff by Disaster Recovery budget spend, overall Disaster Recovery spend, per server Disaster Recovery spend and per user Disaster Recovery spend.  This comparison is performed to give management peace of mind that they are spending the right amount of dollars on the recovery environment(s).   In many cases, its pure disinformation.

Here is why.  Comparing what you spend against other organizations is not indicative that the dollars being spent are prudent, appropriate and  supporting a recovery environment that will meet the needs of your organization and customers:

  • Example 1:  Large multi-billion organization is spending 2% more than their industry annually on multiple backup sites, offsite storage, network, hardware maintenance, staff, etc.  Looks good, right?  The in-depth assessment reveals the budget simply maintains multiple backup sites and strategies that can simply provide for standalone application recovery and not recover the  business.  The spend comparison, even though looking like they spent more than their peers, actually showed dollars were being poorly spent on strategies and technologies that had returned little to no value.
  • Example 2:  Large regional hospital wants to compare themselves against the healthcare industry.  In our experience, hospitals have experienced severe budget constraints over many years and often do not spend the appropriate dollars needed to protect their highly complex computing environments.  So, a comparison against other hospitals may not always be a good guideline.  Lastly, we view hospitals as important as the financial industry.   Hospitals should compare themselves against the leaders in Disaster Recovery, the financial industry.  I always say their “human transactions” are more valuable than “money transactions”.
  • Example 3: Private college in the northeast wants to develop a working recovery environment based on what the industry is spending.  Well, for one, colleges like hospitals, have not been big spenders and two, creating the initial environment will take significant investments over time before spending is somewhat stabilized.

In the many cases we see, the Disaster Recovery environment requires significant investments over time to even reach a stable point of operation before an annual budget stabilizes.  And even then, you could expect a 15% or more budget increase annually with maintenance, new hardware, faster network, storage, etc.  This happens in new and existing recovery programs.  Comparing yourself from a spend perspective can be helpful when you have a stable, working recovery environment that can recover your business.  This allows you to compare your spend more evenly.

Last, don’t always compare to your industry; compare yourself to those who do it best to give you a goal in mind.

Posted in Uncategorized. Leave a Comment »

The Conundrum of Aligning Business DR Needs with Information Technology

We have been involved many times  this year in attempting to align business recovery needs with the IT function.  Many times this effort has been at the behest of the IT organization and / or as part of aligning BIA results with IT for our final report.  As you can well expect, the companies with well aligned Business and IT needs are few and far between; even after completing a “best practice” Business Impact Analysis, there is constant chatter about data validity and what was the business thinking.  This constant disagreement leads to confusion and lack of action.

Additionally, the lack of education at the senior management level regarding current recovery capabilities is beyond comprehension.  We do not expect senior executives to have a “boots on the ground” perspective but at least have a high level understanding of how, when, where and why.  This lack of knowledge is highlighted when we display current capabilities versus needs and you see the deer in the headlights look!    You must educate the executives today, tomorrow and in the future.

To  present the best possible picture for alignment we do the following:

  1. Trust the “best educated” guess of the business unit leaders interviewed.
  2. Use weightings in our BIAs for each impact category to ensure most important categories (loss of revenue, customer service) to the organization receive the appropriate weightings.
  3. Identify the core applications in each BIA to eliminate non-critical supporting applications being deemed critical.
  4.  Include IT representation in all key meetings.
  5. Remind IT this is about the business process and not worry immediately about the servers, network, etc.  needed for recovery.
  6. Have an objective team review the results before presentation.
  7. Remind everyone that manual workarounds have little to no shelf life in today’s highly automated world.
Lastly, we always advise customers that the initial alignment may yield needs that cannot be met immediately.  Developing a plan that will align business and IT DR needs over time is acceptable and in in today’s budget conscious environment, a necessity.
Good Luck!

Characteristics of a Successful Crisis Management Team

Based on past experience, I believe the characteristics of a successful Crisis Management Team are as follows:

  • Supported by Senior Management
  • Proactive in Activation – Not Afraid to Activate
  • Use a Competency Based Leadership Model to Develop Team Members
  • Do Not Manage by Title but by Ability to Lead
  • Roles are Well Understood
  • Team is Scalable Depending on Need
  • Exercises and Training are Regularly Held
  • Depth Across Primary and Backups
  • Team Members Empowered to Act
  • Followed Established Incident Principles and Priorities
  • Follow a Comprehensive Crisis Management Plan
  • Use a Standardized Decision Making Process
These are characteristics we have seen across teams that perform well in exercises and in real events.  What other characteristics do you see as key?
Posted in Uncategorized. Leave a Comment »

IT Change Management – A Key Component to IT Disaster Recovery Success

The need for a comprehensive IT change management process has always been critical but as our technology organizations are resource strained, organizational silos are built, multiple data centers are used, its need is ever more critical.

The size of the organization will not dictate whether or not a good IT change management process is in place. We have seen small IT organizations that have stringent processes in place and Fortune 100 with gaping holes. Or the process is highly dependent on a number of key people who manage the changes like a hawk but when absent changes are made with little resistance.

Without an IT change management process that has teeth when it comes to DR planning, your recovery capabilities stand a good chance of failing when you need them most.

Your iT change management process must address the DR process and ensure changes are reflected in recovery plans, strategies and configurations in parallel with the change being made to production. The simplest of changes could lead to a catastrophic failure in recovery.