My friend Wade Sarver recently blogged about the challenges of large projects. (I’ll link to it in the comments.) His article reminded me of the many Greenfield implementations I’ve done.
A hectic start-up pace, foreign environments, and a team of expats invariably leads to stories. One job was bogged down in internal politics. My company came in with a plan to launch in less than 6 months.
Late Friday night the Tech team were working on the switch interface for billing. This was before TCP/IP had become the de facto standard. IIRC we were tunneling TCP/IP through X.25.
Suddenly all connectivity was lost. We determined the problem must be in the nearby data center. Several of us piled into a car and drove there.
When we arrived our suspicions were confirmed. The problem was in fact in the data center.
A janitor had unplugged AC power to the primary router to power his rotary floor polisher.
We helped him finish up then reconnected our router. D’oh!
After that, we re-designed an #HSRP routing solution including -48v redundant DC power.
👉 Follow me on LinkedIn, Russell Lundberg, for more updates, insights, tips, tricks, and tactics to love a career in Telecoms.