Author: Chris Lange

The following Post will be updated with further details, and will host the formal Root Cause Analysis and steps taken information once its complete.

Starting November 19th until November 25th, SmartHostPlan.com’s Drum Server suffered a major drive failure that impacted the server.  Due to a series of issues, recovery for this server took an extended amount of time that we apologize for.

All primary Resellers, and direct Customers were notified early November 20th about the incident.  Further communication happened directly, via email, or through a new site we setup, https://shpstatus.com.  This new website will serve as the home going forward of our standalone server status page.

Additional details to be posted as soon as full recovery is complete.

Greetings everyone,

SmartHostPlan.com, through automated monitoring and customer contacts, suffered about 3 hours of downtime with the DRUM server earlier today.  Unlike other incidents, this was traced back to the Server Hosting Center and a UPS Power Backup going down.

The issue was identified through monitoring at the Server Hosting Center, and resolved by onsite teams around 10:45 am MDT.

We received a call from our account manager at the Server Hosting Center, and we have validated that the DRUM server came back up and is working as expected.

Thank you,
-Chris Lange

SmartHostPlan.com Upgrade and Maintenance Window for DRUM server, May 2016 (UPDATE 3)

Upgrading is taking longer then expected, but I’m happy to announce that we did successfully install MySQL up to version 5.6.  We’re going to hold off on further updates until this weekend at this point to research an issue with the freetype library.

Updates Completed

• May 1st, 2016
  • Upgrade of the DirectAdmin Control Panel to version 1.5. This was needed to ensure that the backups were completing successfully.
  • Backup of most account.
• May 2nd, 2016
  • Direct backup and download of all MySQL databases
  • Upgrade from MySQL 5.0 the 5.1 (Current advice is to upgrade each version seperatly, so the next upgrade will be to 5.5, followed by 5.6)
  • Updated the MySQL connections and recompiled php 5.4.31
• May 3rd, 2016
  • Another backup of the MySQL Databases
  • Upgrade from MySQL 5.1 to 5.6.  In order to make it work, we did end up skipping 5.5 due to a version error in DirectAdmin.
  • While recompiling php 5.4.31 to work best with MySQL 5.6 we did run into issues with our installed freetype library.  We’re currently investigating what we can do to get beyond the build error.
  • Conducted multiple tests to insure that php and MySQL as they are installed currently are working.  So far so good, but report errors as needed.
  • Update: At 2:30 am we started seeing notices that http was not running.  Diagnostics were ran, DirectAdmin’s forums reviewed, and we are now running on php 5.4.45.  Service was fully restored by 3:05 am.

Next Maintenance Window

• Starting: 2016-05-07 10:00 PM MDT
• Ending: 2016-05-08 2:00 AM MDT

We’re taking the extra time to test and ensure that everything works well. While this is a slower method then planed, upgrading this route ensures that each upgrade works correctly before continuing to the next step.  The upgrades will contain the following planned items:

• PHP upgrade to 5.5
• Apache upgrade to latest in the 2.2 branch
• MySQL upgrade to latest 5.6
• General Yum Upgrade for various server programs

If at any point we start running into issues, we’ll postpone this upgrade until a later date and contact you with the details via our direct customer mailing lists.