Pulled it back from the edge

My work week yesterday had a near disastrous start when one of our main servers had a severe hardware problem.  The server uses a hard drive array to provide protection against drive failure.  In was configured to use a scheme called RAID 5 which means a drive can fail without losing any data.  You are able to swap in a new drive on the fly without so much as a hiccup.

Well this server was showing that TWO drives had failed, the odds of which are very, very slim. In that scenario there is no easy recovery option with RAID 5.  My recovery plan would involve replacing both bad drives, reinstalling the operating system to a base level and the restoring the server via our backup appliance which would take hours and cause a big problem for users on the network that rely on the server heavily.

Well I wasn’t willing to give up on the “failed” drives just yet.  I did a combination of things that wound up allowing me to get one of the two drives that was failed working again.  Once I got that far I could swap in a replacement drive and rebuild it.  Once that rebuild completed I pulled the second drive that was marked bad and let it rebuild as well.  When all the rebuilding was done I added one additional drive in the remaining open slot and assigned it as a “hot spare”, meaning in the future if a drive failure occurs this drive will automatically be fired up to take it’s place.

I felt quite fortunate to be able to get the server back up without all of the hassle and time required for a full server restore.

This morning I dropped Sadie off after her four day stay.  Cindy really liked having her around during the day to keep her company since Sadie is a bit more interactive and expressive than Tuki or the chickens.  It’s always sad dropping her back off but she will back again before we know it.

My moms birthday was yesterday.  I didn’t make mention or note of it publicly.  At some point I feel like I just don’t want to keep giving myself reasons to grieve.