July 2023

By Richard Long, Senior Advisory Consultant, MHA Consulting

[originally published on mha-it.com; written by Colin Garrison based on an interview with Richard Long]

It’s never very hard to find examples of companies whose casual approach to resiliency and crisis management has come back to bite them. Two of the latest are Ticketmaster and Southwest Airlines.

We business continuity professionals spend a lot of time telling our colleagues and clients about the negative impacts an organization can experience if it gives short shrift to the need to become resilient and plan for outages.

We’re also familiar with the tendency of many executives to treat business continuity as an inconvenience and the threat of disruptions as too slight to worry about. Alternately, they might acknowledge the possibility of outages but resist investing in BC on the theory that their people are good enough to improvise their way out of trouble if anything does go wrong.

The recent experiences of two major companies provide a corrective to that way of the thinking.

These helpful negative examples come to us courtesy of Ticketmaster and Southwest Airlines.

Ticket to Nowhere

The Ticketmaster case originated with the popular music equivalent of a hundred-year flood—one going by the name Taylor Swift.

Late last year, tickets went on sale on Ticketmaster for Swift’s Eras Tour, her first in five years. The company counted on its Verified Fan program to weed out bots and speculators and ensure that actual fans would have a fair chance to buy tickets. However, the moment tickets went on sale, the company’s system was overwhelmed, causing millions of fans to spend hours in limbo and resulting in the eventual cancellation of the sale.

The company said it received 3.5 billion system requests that day, four times more than its previous high, according to the New York Times.

The debacle led to the CEO of Ticketmaster’s parent company, Live Nation, being grilled before a Senate hearing earlier this year. He apologized to Swift and her fans and promised to do better in the future.

More consequentially, perhaps, the incident also awoke a sleeping dragon in the form of renewed bipartisan interest in reigning in the company’s dominant—some say monopoly—position in online ticket sales. (Some senators said the company’s dominance had led it to grow complacent, stop innovating, and take its customers for granted.)

I am inclined to give the company the benefit of the doubt when they say they were simply unprepared for the unprecedented level of demand. What struck me about Ticketmaster’s response to the incident were two things: how unprepared it was for an outage of this type and the clumsiness of its initial public response.

It is now clear that every technology company is going to experience a meltdown at some point. However, even if the collapse of Ticketmaster’s servers under the Swiftie onslaught can be excused, the company’s apparent lack of preparation for a generic outage is hard to explain away.

Equally bad was its early public relations effort, which ranged from indifferent to insulting. The company didn’t seem to care about the impact of the outage on the millions of people who were vying for the chance to be its customers. The company’s apparent arrogance splashed gasoline on the fires of the Swifties’ frustration and disappointment, making a bad situation worse.

MHA Consulting CEO Michael Herrera and I often write in our blogs about how important it is for companies to prepare and vet scripts ahead of time for use in foreseeable crisis situations. It seems obvious based on the clumsiness of its initial response to the Eras Tour outage that Ticketmaster had no such scripts. This lack of preparation and the problems it caused is a great negative example showing the need for companies to compose advance scripts for foreseeable types of problems.

Stuck on the Ground

Another recent negative example about the high cost of inadequate preparation is Southwest Airlines, though in fairness they also did some things quite well in responding to their service outage.

On a weekday morning in April, the carrier was compelled to halt flight operations nationwide for 30 minutes so it could work through a computer problem involving data connection issues and a firewall failure. This led to a delay of almost 2,000 Southwest flights.

Like the Ticketmaster meltdown, Southwest’s outage attracted the attention of government officials. The Secretary of Transportation tweeted that his agency was watching to ensure the airline took care of all passengers impacted by the outage, and the chairwoman of the U.S. Senate Committee on Commerce, Science, and Transportation said, “This is another demonstration that Southwest Airlines needs to upgrade their systems and stop the negative impacts to individual travelers.”

In this case, Southwest’s systems and processes were not robust enough to allow them to make adjustments that would have lessened the impact of the outage. In some cases, they had crews sitting around after their flights had been cancelled and other planes ready to go but with no crew; however, they lacked the ability to match up these available crews and planes. 

In the big picture, Southwest’s troubles stemmed from a continuing staff shortage and a failure to plan for the possibility that multiple challenges can occur simultaneously (in this case, a staff shortage and a system outage). (For help on dealing with this issue, see Michael Herrera’s post, “Double Trouble: How to Handle Multiple Business Disruptions.”)

On the positive side, Southwest’s response to the outage was admirably transparent. In communicating with its passengers about the delay and helping them reach their destinations, the airline was empathetic, candid, and proactive—all the things Ticketmaster was not.

Avoiding Damaging Consequences

The recent experiences of Ticketmaster and Southwest Airlines serve as cautionary tales for businesses on the importance of resiliency and crisis management. Ticketmaster’s disastrous handling of the overwhelming demand for Taylor Swift tickets exposed their lack of preparation for outages and revealed a clumsy initial public response.

Southwest did much better in the public response to their system outage and flight delays, but their experience revealed the need for organizations to be prepared for simultaneous business disruptions. These examples underscore the significance of preparedness and effective crisis response in maintaining customer satisfaction and avoiding damaging consequences.