Call to get started(502) 440-1380

The Boiled Frog

Many IT departments in small and mid-sized companies run into resource challenges. These resource challenges take on many faces, but regardless of where the IT department is constrained this week, the level of detail internal staff can get to and have a life outside of work is often minimal.  I have seen only a few talented IT generalists keep many balls from falling on the floor. But eventually, the juggler gets sick or tired or both and a ball or two end up on the floor and often in a very dramatic and painful way.

The attention to detail is a necessary quality an IT department needs to have. With all of the various fires that need to be put out on a day in and day out basis, simple maintenance and documentation tasks fall to the wayside. Here is an actual example.

The IT department in a mid-sized company helped architect a new financial management IT system for their company.

PeopleSoft (now Oracle) was the chosen vendor. The IT department took the hardware requirements from PeopleSoft and implemented them to the letter. Everything functioned fine.  About 3 years later, however, the system had a major crash due to a power outage with lost data and the already resource-constrained IT department worked around the clock for weeks to recover from it. The root cause was not the design or the system itself. It was the lack of maintenance and attention to detail.

Post-Mortem:

There were several issues that arose from this lack of attention to detail and routine maintenance tasks.  A few of the key issues included an over-subscribed UPS and unsupported versions of software and hardware.

With the UPS, regular tests were not performed and checking the battery viability did not happen. Additionally, over time, other servers were plugged into the device that was designed to only support the financial system. This resulted in a 30-45 seconds of battery life and when that time was up, all the servers crashed hard.

After several of the other issues were addressed after the crash, the team needed to restore data from back-up tapes. Because the back-up process was not monitored well and restoration tests were never performed, the team didn’t discover that the version of back-up software was no longer supported by the manufacturer of the tape drive until they needed to restore critical data.

Mature IT processes and the discipline to make sure that the processes are followed is the key difference between what Principium Technologies offers and what IT departments can usually provide. It is not the IT departments’ fault either. Fires take precedence, followed by project work and when that takes up more than 50 hours a week for each IT employee, they opt to let this maintenance lapse. But just like the boiled frog, when the water slowly reaches that critical boiling point, an IT department ends up on the bad side of a crisis. Principium Technologies pays attention to the little things so that your IT department doesn’t have to.

Share this Article

Plugged In

View all articles