Could Your Business Handle Another Hurricane Sandy?

Looking back at the aftermath of Hurricane Sandy, it’s hard to believe that what began as a pocket of low pressure Caribbean air would turn into one of the largest Atlantic-based hurricanes ever recorded. By the time Hurricane Sandy reached New Jersey as a post-tropical storm with hurricane-force winds, the state was a sitting duck for the intense flooding and widespread damage that would ensue.

Now, one year and $100 billion later, businesses are finally in a position to look back and reflect on ways of being better prepared to handle such a disaster.  While the loss of life and memories from Hurricane Sandy can never be erased, it is now possible to learn from the past and come up with new ways to prevent such widespread catastrophe from happening again.

Unfortunately, many businesses located along the Eastern seaboard are still in harm’s way.  As businesses have slowly begun the reconstruction process, there is still the ever-present threat of another major storm arising.  This begs the question: if Hurricane Sandy was to happen again, would your business be capable of surviving?

Lam Cloud offers everything your business needs to thrive in the event of an unexpected disaster.

The company’s Cranbury, NJ Technology Campus comes equipped with flexible Disaster and Workplace Recovery resources to help your business function in the event of a crippling disaster – whether natural or man-made.  The facility features a backup generator capable of kicking in within 12 seconds as well as both UPS  dedicated A/B power feeds to ensure fault-tolerant data protection at all times.  Additionally, Lam Cloud provides redundancy and resiliency through raised flooring built above a 500-year flood plain.

For those displaced from their regular workplace, Lam Cloud offers Work Area Solutions, including 4,000 Workplace Recovery seats, meeting rooms, executive technology offices, and a Tech Lounge for relaxation and entertainment – ensuring your staff is comfortable and productivity is not jeopardized in the temporary work environment.  The 500,000-square foot facility also offers ample Disaster Recovery resources, encompassing custom client solutions, VoIP services; equipment provisioning, secure backup and archiving, server and desktop virtualization, Ethernet and Internet access, and carrier-neutral connectivity.

For companies seeking colocation, Lam Cloud owns and operates a 70,000 square feet of SSAE 16-compliant, scalable data center space with capacity for 1,200 server cabinets, with an average IT load of 6.0 kw per cabinet.  Additionally, the organization offers a comprehensive suite of Professional Services such as business impact analysis, gap analysis and audit, business continuity planning, and proof of concept testing.

For more information about Lam Cloud’s Disaster and Work Area Recovery offerings, please click here.

To view Lam Cloud’s Data Center Discovery profile please click here.

FORTRUST celebrates 12 years of 100% uptime at Denver data center

In the world of data centers, uptime is the name of the game. The difference between a successful data center and a tragic failure isn’t a spooky-low PUE, massive 2N+ infrastructure or a precious metal LEED certification.

These features and metrics are certainly valuable tools and have their place. However, no one gives much thought to data center efficiency, design redundancy or sustainability if the facility is prone to unplanned downtime.

So, earlier this year, when FORTRUST announced that their 300,000 square foot, 30 MVA Denver data center had just passed 12 years of “continuous critical systems uptime without a single instance of unplanned downtime” people took notice. The question on everyone’s mind was simply, “How?”

FORTRUST Denver Data Center

Fortunately, FORTRUST’s SVP/GM, Robert McClary doesn’t mind telling how he and his team have been able achieve this unprecedented benchmark in data center operational continuity. In fact, FORTRUST and McClary have just released an e-book titled “A Data Center Operations Guide for Maximum Reliability” that goes into depth describing the challenges and strategies involved in successfully operating a high availability data center.

McClary starts his paper by addressing the persistent misconception that robust data center design leads unerringly to high availability.  He states, “…it is my experience that the design is only one small factor in the equation that results in continuous uptime. The larger factors contributing to high-availability and uptime are specific to people, process, operations, maintenance, lifecycle, and risk mitigation strategies.”

From this starting point, McClary gets into the meat and potatoes of his data center management philosophy. He systematically knocks down the excuses that many companies use to avoid rigorous operational standards. Here are a few:

  • Don’t have the time, money or talent to write procedures for maintenance practices? Better find some.
  • Personnel don’t feel they need a procedure? Better fix that thinking.
  • No time for training? Better make time.
  • Labeling infrastructure is hassle? Deal with it.
  • DCIM software is expensive? Not compared to downtime.
  • And my personal favorite; can’t find qualified personnel? Hire a veteran.

McClary backs up each of his points with logical, no-spin reasoning. It’s clear that he’s spent a lot of time thinking about and practicing strategies that produce actual results. His writing rings with experience, clarity, authority and a no-nonsense approach to mission critical facility operation.

McCLary is blazing a trail to radical uptime through operational excellence. His message to those who would duplicate his success is clear: Here’s the method. Follow me…if you can.

FORTRUST is the premier high availability data center service provider in North America offering services in Denver, Colorado; Phoenix, Arizona; and Edison, New Jersey. FORTRUST offers agile, reliable, sustainable and secure raised floor or modular data center capacity for any-size enterprise supported by optimal power infrastructure and connectivity to safeguard mission-critical business services. Leading companies choose FORTRUST to gain a trusted partner who will preserve and protect their IT infrastructure as well as serve as an essential extension of their operations.



The Threat of Cyber Attacks on Data Center SCADA Systems

Editor’s note: I originally wrote this post two years ago for another blog called datacenterpro. Datacenterpro primarily deals with Low-Probability, High-Consequence (LPHC) types of threats to data center operations. So, you will see posts there about topics such as geomagnetic storms and cyber-security threats. This post received nearly 10,000 hits, was picked up by MIT’s Technology Review blog and led to a cover article for Mission Critical Magazine.

I’m reprinting it here because I’m a shameless self-promoter and need to draw web traffic to my new website because it may be of interest to new readers here at Data Center Discovery. Happy reading!

Earlier this year, shortly after the discovery of the STUXNET cyber weapon, I wrote an article for Mission Critical Magazine (MCM) titled, “The SCADA Worm Threat to Mission Critical Infrastructure”.  In the article, I explained how STUXNET had demonstrated a new and profoundly dangerous threat to Industrial Control Systems (ICS) such as SCADA (Supervisory Control and Data Acquisition).  I urged the data center community to recognize that data center electrical and mechanical infrastructure is potentially vulnerable to this type of cyber attack.  Furthermore, I warned that data centers can be very enticing targets for criminals, terrorists and foreign nationals utilizing cyber weapons.  I predicted that SCADA cyber attacks in general would become more common and that data centers could become targets for data theft, extortion and sabotage through their SCADA systems.

Since the release of the article, a number of trends have reinforced my view that SCADA systems are becoming increasingly vulnerable.  Events of concern include:

  • Proliferation of STUXNET SCADA worm technology.  On September 1, 2011 a new SCADA worm, dubbed Duqu, was discovered.  The Duqu worm bears close resemblance to the STUXNET worm in complexity, design and execution.  However, Duqu was configured for a completely different (and currently unknown) target.  Initial analysis indicates that Duqu may be designed to steal data as a precursor to a STUXNET type cyber attack.   The similarity to STUXNET indicates that Duqu’s designers either designed STUXNET or had access to the STUXNET source code.
  • Rise of hactivist interest in ICS cyber attacks.  In September of 2011, a US Department of Homeland Security (DHS) bulletin provided evidence that the hacking collective “anonymous” “had recently expressed an interest in targeting industrial control systems (ICS).”  It is doubtful that anonymous will have the capacity to execute a STUXNET level cyber attack in the near future.  However, their interest in exploiting ICS technology is indicative of an increase in awareness and activity within the hacking community regarding ICS systems.
  • SCADA hacking malware (almost) demonstrated at TakeDownCon In May 2011, security researchers from NSS Labs were planning to demonstrate how to write “industrial-grade” SCADA malware at a Dallas information security conference.  The researchers claimed, “We will demonstrate how motivated attackers could penetrate even the most heavily fortified facilities in the world, without the backing of a nation state.”  SCADA manufacturer Siemens and the US Department of Homeland Security requested that the researchers not continue with the demonstration citing public safety concerns.  The NSS Labs researchers complied with the DHS request.
  • Additional SCADA vulnerabilities made public.  In March 2011, security researcher Luigi Auriemma posted full-disclosure advisories and details regarding proof-of-concept attacks for thirty five new SCADA vulnerabilities.  Auriemma posted these to the publicly available (Bugtraq), an open bulletin board for Symantec customers, end users, developers and partners.
  • Powerful SCADA Hacking “Toolkit” released.  In March 2011, Gleg, a Russian security firm offered for sale a software package known as The Agora SCADA+ Pack.  The software contained 22 modules exploiting 11 zero-day vulnerabilities.  The pack included data applicable to a wide variety of SCADA system manufacturer’s devices and software.  The package also allegedly contains analysis of SCADA system “weak points” such as hard-coded passwords and problems with smart chips.

Clearly, cyber criminals are beginning to focus their attention on SCADA systems and are busy developing new exploits and malware

The vulnerability of SCADA systems represents a particularly grave threat to infrastructure of national significance.  Vital infrastructure such as electrical grids, refineries, water treatment plants and chemical processing plants rely heavily on ICS and/or SCADA.  The consequences of a successful cyber attack on this infrastructure are potentially dire.  Fortunately, some national governments have recognized that the SCADA cyber vulnerability represents an emerging threat to national security and have taken steps to close security gaps.   The US Federal Government, for example, has launched extensive cyber security initiatives and programs to address vulnerabilities in our national infrastructure.  US-CERT, a division of the US DHS, has become one of the world’s leading cyber security organizations.

SCADA systems are not limited to industries of significance to national security.  In fact, variations of these systems can be found in nearly every industrial and commercial environment.  Data centers are no exception.  Most commonly, data centers utilize SCADA technology to control the automated functions of their critical electrical switchgear.  Switchgear in these facilities usually feature multiple, redundant power paths to allow for maintenance and to provide operational resilience in the event of a system component failure.  In order to function effectively, this type of switchgear must monitor system conditions such as voltage, amperes and frequency.  If one of the monitored parameters falls out of a preset tolerance band the switchgear automatically performs an action or series of actions to correct the abnormal condition.  For example, in the event of a loss of mains power to the switchgear, standby generators start and a number of circuit breaker position change in order to deliver generator power to the critical load.  The system of sensing devices, Programmable Logic Controllers (PLCs), and computers that monitors and controls the switchgear is known as SCADA.

For many years, data centers and other users of SCADA systems operated without significant threat from hackers, malware and cyber criminals.  These systems benefitted from a flawed security principle known as security through obscurity or hiding in plain sight.  SCADA systems utilize communication protocols (for example MODBUS) that are not widely known by hackers and malware developers.  Furthermore, the systems monitored and controlled by SCADA are often extremely complex and require extensive training to understand and operate.  It was considered unlikely that an intruder in the system would have the engineering knowledge needed to effectively infiltrate the system and cause lasting damage.  These system characteristics amounted to a degree of obscurity that did not seem to require extensive cyber security.

The security of SCADA systems also benefitted from a persistent question of motive.  Hackers and malware are typically associated with the theft of sensitive corporate secrets, personal information or financial data.  This type of data is not stored in Industrial Control Systems.   Thus, ICS manufacturers and operators assumed that their systems would not be hacked because they contained no data that might justify the work required by a hacker.

Additional security was assumed because SCADA systems are not typically connected to the Internet.  However, these systems are routinely accessed for software upgrades, data exports and system configuration changes.  Additionally, many SCADA systems share network infrastructure with other corporate networks.  This practice allows the SCADA system to share data with other corporate assets and avoids the cost of a separate, dedicated network for the SCADA system.  However, these practices compromise security integrity for these systems.

Given the assumed security through obscurity, the lack of traditionally targeted data content and the lack of direct Internet connection the primary security threat to SCADA systems appeared to come from accidental misuse by poorly trained operators or deliberate misuse by disgruntled employees.  The solution to this type of security problem consisted of restricting access to the SCADA controller using rudimentary (usually default) passwords and physical security.

In 2010, the appearance of STUXNET shattered the illusion of security for operators of SCADA systems.  The STUXNET cyber weapon was a piece of malware (specifically a worm) which was engineered to target a uranium purification facility in Iran.  The STUXNET worm utilized USB drives and autonomous replication capability to infect the SCADA system in the highly secure facility.  The systems were infected despite the fact that they were not connected to the Internet.  Once inside the system, the malware cunningly hid itself in system memory, reprogrammed Programmable Logic Controllers (PLCs) and sent false data to the system SCADA controller or Human Machine Interface (HMI).  The new PLC programming caused momentary speed changes in the high speed uranium purification centrifuges in use at the facility.  These speed changes had the combined effect of rendering batches of purified uranium unusable and causing catastrophic physical damage to the centrifuges.  The net effect of the attack was to set the Iranian nuclear power program back by years.  When the worm was finally discovered months after its payload was delivered, the international cyber security community promptly labeled STUXNET a “game changer” and the first “cyber super weapon”.

For the first time, malware had been successfully deployed against a SCADA target and caused catastrophic physical damage to the controlled system.  Clearly, the obstacles of obscurity and complexity could no longer be counted on to keep SCADA systems secure.  The creators of STUXNET had demonstrated that these obstacles were irrelevant to highly motivated and educated malware developers.  Clearly, a lack of Internet connection could no longer be considered adequate protection for SCADA systems. Trojans, worms and other malware can infect SCADA systems via secondary network connections and via devices used to perform necessary maintenance tasks.  Clearly, the question of motive was answered.  SCADA technology had been adopted by so many critical industries that abundant motive could be found to justify building the tools needed to crack these systems.

The complexity and sophistication of the STUXNET worm indicated that it was the work of a national intelligence agency.  However, many cyber security professionals began discussing the longer term ramifications of the existence of such powerful SCADA worm malware.  Drawing on their experience with the development and spread of conventional worms and viruses, experts warned that now that this type of weapon had been deployed, the techniques and source code would be replicated and repurposed by a widening array of cyber criminals.  Because SCADA technology can be found in nearly every industrial environment and because these systems usually lack even rudimentary cyber security features, experts warned that attacks on these systems would quickly become commonplace.

Some cyber professionals argued that operators of SCADA systems that are NOT part of the national infrastructure are actually at greater risk than targets of national security significance.  Ralph Langner (the man who “solved” STUXNET) of Langner Communications, warned that cyber criminals using SCADA worm malware would avoid public infrastructure targets in favor of poorly protected private enterprises with sizable financial resources.  Langner predicted, “The next cyber weapon will be considerably cheaper, since much of the attack vector and the specifics of how to use automation equipment will simply be copied.  Sabotage with the motivation of extortion will get a commonplace scenario.  At this time targets are no longer limited to critical infrastructure but will especially cover the private sector — a TARGET-RICH AREA where it cannot be assumed that organizations will install countermeasures large scale in a reasonable amount of time.”

Fortunately, some private industries are actively hardening their SCADA infrastructure against cyber attack.  Many electric utilities, chemical manufacturing plants, water treatment facilities and oil & gas infrastructures, at the urging of the U.S. Congress and in cooperation with agencies such as US-CERT, have taken many steps to secure their systems.  In addition, a number of professional cyber security firms have emerged to specifically address SCADA vulnerability for these industries.  However, the data center industry has largely been slow to implement meaningful security measures.  Ironically, an industry that is profoundly conscious of the cyber security threats aligned against the IT assets on the raised floor seems to be unconcerned regarding the security issues relative to the SCADA in the facilities space.

In the current political and cultural climate there are a variety of groups that may develop the motive and skill to target data center SCADA infrastructure for cyber attack.  These groups include:

  • Nations engaged in cyberwar.  In 2007 a Blue Horizons paper, titled, “State Actor Threats in 2025” was prepared by the US Air Force.  The paper identified a number of scenarios that could threaten the United States in the future.  The scenario with “the highest potential for a state actor to inflict catastrophic damage to the US” is known as Phantom Menace.  In this scenario, cyber attacks are used, “against the enemy so that the civilian electricity network, traffic dispatching network, financial transaction network, telephone communications network, and mass media network are completely paralyzed, this will cause the enemy nation to fall into social panic, street riots, and a political crisis.”  Each of the targeted infrastructure assets identified could be crippled by attacks that shut down the data centers that control those industries.
  • Corporations and nations engaged in industrial espionage.  In 2010, Google revealed that for the second half of 2009 it had been under constant cyber attack.  Security professionals at McAfee named the attack Operation Aurora and identified the attacks as an advanced persistent threat (APT), (a classification of attack that also includes the STUXNET malware.) Google indicated that the cyber attack originated in China.  Operation Aurora was not limited to Google assets but also included assaults on other major American companies.  Adobe Systems, Juniper Networks and Rackspace have publicly confirmed that they were targeted. According to media reports, Yahoo, Symantec, Northrop Grumman, Morgan Stanleyand Dow Chemicalwere also among the targets.  In an era where state actors can attack public companies using cyber weapons, it is not inconceivable that data center infrastructure could be jeopardized.  Nations and companies could gain competitive advantage over their adversaries by disrupting operations at their data centers.
  • Cybercriminals targeting data center infrastructure for purposes of extortion.  As Ralph Langner pointed out, malware has become a common weapon used by criminal organizations.  As SCADA cyber weapons proliferate, it is expected that the technology will find its way into the hands of criminals that will use the technology to extort funds from corporations.  In my article in MCM, I outlined a possible scenario where a SCADA worm could be used to destroy an emergency generator at a data center. The damage would be followed up with a threat of more damage unless a hefty extortion threat is complied with.
  • Social activists seeking to disrupt credit and banking infrastructure.  The news during the autumn of 2011 was dominated by stories of “Occupy” protesters in major American and European cities.  Fortunately, these protest groups lack a cohesive political message or effective leadership.  However, these groups represent a general rise in antipathy toward banking and commerce organizations.  A cyber attack on commerce infrastructure such as a stock exchange or credit card processing data center would meet the apparent aims of these groups.  The Occupy protesters may find support for such an attack from environmental activists who view data centers as major consumers of “dirty” electrical energy.

The trend regarding SCADA attacks is clear.  The weapons used to perpetrate these attacks are becoming more widely spread and more powerful.  Simultaneously, the expertise and techniques required to successfully deploy these weapons is becoming more common.  Finally, the number of groups that could benefit from deploying one of these weapons against a data center is increasing.  Each of these trends points toward a bleak future for the unprepared data center.

EnSite Solutions achieves staggering growth through great customer service

Earlier this month I got an opportunity to talk with Toby Thomas at EnSite Solutions. EnSite Solutions provides preventive and remedial maintenance services for mission critical HVAC, UPS, Battery, Generator, and Fire Suppression systems. They provide these services across the entire nation and they boast the most factory-trained field technicians in the country.

At a time when other SMBs in the mission critical sector are nervously looking at shrinking revenues and dwindling markets, Toby has achieved staggering growth at EnSite.

Their growth has been so remarkable that in October, Inc. Magazine named EnSite Solutions to their annual “The 2013 Inc. 5000 List”. The Inc. list identifies the fastest growing companies in the United States. EnSite Solutions came in at number 187 on the list for achieving a mind blowing 2,207% revenue growth over 3 years.

I asked Toby how he was able to steer EnSite to such incredible growth. Not surprisingly, it seems that the answer is; really great customer service.

Toby told me, “EnSite enables our customers to take part in our national network of top tier partners in the HVAC, UPS, Battery, Generator, Fire Suppression services.” In addition, EnSite customers can, “tie into the service offering our customized web portal…to complete the fully customized service platform.” Finally, EnSIte provides their customers with “an average response time of 1.2 hrs from time of call to being onsite.”

In a nutshell; top tier services, web based service management and crazy fast response times. This formula has led EnSite to, “an unheard of zero customer churn rate in the last five years.”

We are proud to have EnSite Services in the Data Center Discovery directory and are looking forward to seeing what this customer focused company will achieve next.