Finally coming up for air

Wow I just went through one of the biggest cluster f’s of my IT career.

We were scheduled to perform an upgrade to the software we use for cashiering on Wednesday night.  You would think that the upgrade wouldn’t be that big of a deal since we were moving from version 2.37 of the product to version 2.45.  Well you think wrong.

When this upgrade was planned out we estimated it might take 2-3 hours max.  It was an after hours upgrade that we started on at 5pm.  I left the office at 12:30 am.

Without boring you with technical details, basically everything you could imagine going wrong did.  I lost count of how many technical hurdles we had to clear.  By the time we left we thought we had the core product working with some ancillary issues that we would deal with the following day.

Well when I arrived bleary eyed the next morning after a few hours of sleep it turned out there was a fresh new boatload of issues to hammer out, stuff we couldn’t have tested until clerks started doing stuff.  I spent nearly the entire day on the phone, sometimes with two or three people simultaneously working on both the problem and putting out fires simultaneously.

Finally late afternoon we figured out what was the root of our problem and corrected it.  There are still some minor things to squash of course.

Some of my co-workers commented that they couldn’t believe how calm I remained throughout what was an incredibly challenging and frustrating situation.  Despite my many personality deficiencies I am pretty damn good at fixing problems.  I learned long ago that panic does not assist in that process.

The timing for this was not great as at home we have all sorts of ducks to line up for the half marathon I am timing this weekend.  Registration for this year has swelled to the highest numbers since I have been involved with the club.  I have a bunch of people wanting my attention regarding various aspects of the race.

In addition to helping with issues before the race Ali is going to be running in the event as well for the second time.  She hopes to improve on her time from my last year. In a way I am jealous, now that I have the idea of completing a half marathon in my head.

After this event we still don’t get any breaks as we move right into preparing for Ali’s race two weeks later. Just gotta keep moving.