Bucket brigades of diesel fuel carried up six flights of stairs, 5 and 10 gallons at a time. Provisioning of three different backup generators. Employees sleeping in a cold, dark office.
Those were just a few of the steps MLB Advanced Media undertook in the four days after the Oct. 29 arrival of Hurricane Sandy to keep alive MLB.com, ESPN3, political pundit Glenn Beck’s GBTV, in-flight entertainment company Row 44 Inc., and several other MLBAM-powered digital video services and websites.
 |
Employees keep one of MLBAM’s generators fueled up.
Photo by: MLBAM |
MLBAM, based in Manhattan’s Chelsea Market, lost electric power for about 100 hours as much of greater New York grappled with outages. Many other technology and digital media companies based in lower Manhattan either suspended service altogether after the storm or published on a limited basis, including the Huffington Post and Gawker Media, but MLBAM kept itself and its growing battery of third-party clients fully operational.
In total, the company delivered more than 500 hours of live video programming without the benefit of city power, including the San Francisco Giants’ World Series victory parade.
Doing so, however, required a continual reshuffling of backup power and resources. The first step was using an existing 600-kilowatt backup generator on the roof of the Chelsea Market. The size of a small car, the unit burns through 400 gallons of diesel a day, which, because of the compromised conditions, was ferried up the building stairs by bucket brigade, including by flashlight after nightfall.
But that generator is designed to run for up to 72 hours continuously before needing cooling time and servicing. So as soon as that first generator was activated and it became apparent the city power outage would last at least several days, MLBAM executives began searching for replacements.
The company struck good fortune by finding an available commercial generator designed for use in remote TV productions, and then a second, similar one later in the week. With the assistance of longtime company aide AMA Consulting Engineers, MLBAM was also able to secure more diesel fuel to keep the units running.
But even with the steady hum of generator engines running, MLBAM continually was recalculating and rationing its power needs with an eye toward extending the fuel efficiency of the generators. Transitioning the collective power draw from one generator to the next, and then back to main power once it was restored, without compromising any of the online products, presented more hurdles.
“I was constantly running through joule calculations,” said Joe Inzerillo, MLBAM senior vice president of multimedia and distribution. Inzerillo’s facility background as the former chief technology officer for the United Center in Chicago proved useful — he essentially spent the week as an electrical engineer. “Every single draw of power had to be monitored and measured, even down to charging cellphones.”
MLBAM kept the office active with a skeleton crew of roughly two dozen staffers from its production, engineering, multimedia traffic, IT support and office services departments. All others in the 600-person staff were told to stay home until power returned. Chelsea Market itself was closed to the public.
Well before the storm, Inzerillo and other MLBAM executives were developing plans to create a twin multimedia data center to replicate the one in New York and serve as a backup in crises. That effort is accelerating, with locations being scouted in the Midwest and a new center likely to open before Opening Day 2013. Creating the second center will cost at least $10 million.
“We got lucky this time,” Inzerillo said.
