12:59 PM 10 45.1 83% pp Chronology Of An Outsourcing Disaster Austin Energy's deal with IBM for a new billing system has gone painfully awry, internal emails reveal, resulting in lost revenue for the utility and outraged customers. By Paul Mcdougall, InformationWeek February 23, 2012 URL: http://www.informationweek.comews/services/outsourcing/232601344 IBM's contract to provide a Texas utility with state-of-the-art billing systems has sunk into such a quagmire of missed deadlines, buggy software, and costly errors that the utility's CIO said he's gravely concerned" about whether IBM can fix the problems. "We have yet to reach a stable system," said Austin Energy CIO Alan Claypool, in an email to an IBM exec dated Feb. 10. "We are extremely disappointed and continue to have serious concerns about the quality of service we have received from IBM to date." Claypool's message followed months of correspondence between himself and his staff and IBM project managers on multiple problems. At one point, Austin Energy's general manager asked then IBM CEO Sam Palmisano to personally intervene, but Palmisano passed the request to a subordinate. Austin Energy contracted IBM in 2009 to build a centralized billing system for the electricity, water, and waste disposal services it provides to more than 400,000 residents of the Texas capital. The contract called for Austin Energy to pay IBM a total of $55 million over eight years, but already the utility is holding back $3.8 million in scheduled payments for what it says is IBM's poor performance. The billing system, built on IBM Websphere and Oracle database technologies, was supposed to go live in early 2011, but it's still not fully operational. Meantime, Austin Energy says it has lost significant revenue because its customers have been receiving erroneous bills or no bills at all. A business customer that should have been billed $3,000 got hit with a bill for $300,000, according to a company spokesman. Problems date back more than a year, emails obtained by InformationWeek through an open records request show. In a Sept. 7, 2011 message to IBM VP Robert Hallman, Claypool complained that IBM was repeating mistakes as it tried to implement the system. "During the recent Dress Rehearsal 2, two separate errors by IBM cost the City project a combined total of 37 hours of delay," Claypool wrote. "One of the errors was the same type of error made by the same team in December 2010. The second was by an employee who was not available for executing a key task at the previously scheduled time." Claypool added: 'These types of issues continue to be a part of the ongoing concerns with IBM's quality of delivery." Ten days later, things weren't getting any better. "Again, we continue to be severely disappointed by the implementation services from IBM," Claypool wrote to Hallman on Sept. 17. "As you are aware, we had an unscheduled outage while teams were preparing for our major testing this morning." [ Implementation of complex software is often dificultad 18-Month Rollouts: Nothing To Brag About. ] Claypool added: "As a result of the repeated errors and delays by IBM, we continue to be gravely concerned at IBM's ability to complete a successful system integration and go live." On Sept. 27 of last year, Claypool told Hallman: "We continue to be gravely disappointed at the delays and seemingly ad hoc methods toward managing this project." That same day, Kerrica Laake, an Austin Energy division manager, informed Claypool that a portal IBM set up so customers could pay bills online wasn't working. "IBM has failed to properly set up and fully test their payment solutions for the Online Customer Care Portal," Laake wrote. Frustrated with the mounting problems, Claypool wrote directly to Marc Lautenbach, who runs IBM's Global Business Services unit in North America, on Oct. 17. "As the City of Austin continues to face seemingly endless failures and crashes by our IBM 'partners' on the current and past implementation projects, I find myself writing you, in hopes that you will, at long last, be that person. Beyond that, I don't even know what to say at this point," Claypool wrote. Lautenbach responded the same day: "Alan, I'm aware of the situation and I am disappointed we are not meeting your expectations ... We will get back to you with a plan." Things annarently had not antten any hetter 11 dave later when Clavnonl fired off the following12:59 PM 10 4Fall 83% Lautenbach, who runs IBM's Global Business Services unit in North America, on Oct. 17. "As the City of Austin continues to face seemingly endless failures and crashes by our IBM 'partners' on the current and past implementation projects, I find myself writing you, in hopes that you will, at long last, be that person. Beyond that, I don't even know what to say at this point," Claypool wrote. Lautenbach responded the same day: "Alan, I'm aware of the situation and I am disappointed we are not meeting your expectations ... We will get back to you with a plan." Things apparently had not gotten any better 11 days later, when Claypool fired off the following missive to IBM managing partner Jay Bellissimo. "Jay, when we contracted with IBM for implementation services, we did not plan for, staff, or anticipate using our system and folks as the Alpha testers for the IBM code deliveries. Prior to delivery, we expect the code has been reviewed, validated, and will perform without software defects." With little progress by late November, Austin Energy general manager Larry Weiss appealed directly to Palmisano on Nov. 22, 2011. "Instability issues have been substantial and continue to have serious and costly impacts to our business and our customers," Weiss wrote. "In short, the City of Austin cannot properly operate its utilities. We are losing revenue." Weiss said system errors meant Austin Energy couldn't bill apartment residents for water, balance its books, or file audit reports. "We have tens of thousands of customers who have required one-on-one assistance either in accessing their accounts through the portal or in addressing account difficulties resulting from system instability or error. This results in substantial detriment to customer relationships, our reputation, and work efficiency capability," Weiss said. "My concern is the lack of progress in identifying the root cause and, frankly, the pace and quality of response from IBM." Shortly afterwards, Austin Energy officials voiced concerns that some of IBM's proposed fixes, such as adding more server capacity, would mean that the utility would have to pay more than it originally planned on the project. "IBM implies there will likely be additional cost to the City as a result of IBM's decision--as part of the recovery effort--to move to more powerful servers," wrote Austin Energy's Kerry Overton to IBM's Chris Lilley on Nov. 23. "We do not support additional cost for the replacement architecture under the fixed price contract." Soon after, Frank Kern, then head of IBM Global Business Services, wrote to Austin Energy GM Weiss to inform him that Palmisano had handed him Weiss's complaints. "Sam Palmisano has asked me to respond to the concerns raised in your Nov. 22nd letter," Kern wrote on Dec. 8. "I first want to assure you that the IBM team is equipped to handle the issues that typically arise in complex billing solutions like the one at Austin Energy, and we intend to meet IBM's contractual commitments." Kern outlined five steps IBM would take. They included "improving communications on business impacts caused by known defects," ensuring that all outstanding issues were documented and assigned to the right people, implementing "best practice processes to ensure repeatable success," working more closely with third-party vendors such as Oracle, and "identifying any remaining gaps that are outside the project scope and providing recommendations." Kern announced his retirement earlier this year as IBM announced management changes under new CEO Ginni Rometty, who succeeded Palmisano on Jan. 1. Rometty tapped former North American sales leader Bridget Van Kralingen to head Global Business Services. Claypool wrote to Van Kralingen earlier this month, complaining that little had changed since Kern proposed his remediation program: "The City of Austin continues to sustain substantial financial and operational losses resulting from two multi-million dollar procurements in which IBM has failed to provide the fully functional systems required out of three major system integration contracts we have with you." Claypool was referring to the billing system and also to an inventory management system IBM was supposed to build for the city. "More than a year later, the City is still suffering from serious, continuing issues in getting the system running and using it to conduct routine City of Austin business," Claypool wrote. As for the billing system, Claypool told Van Kralingen that Austin would hold back $3.8 million in payments to IBM until the situation is O12:59 PM 10 4Fall 83% success, working more closely with iniru-party vendors such as vracie, and luenulying any remaining gaps that are outside the project scope and providing recommendations." Kern announced his retirement earlier this year as IBM announced management changes under new CEO Ginni Rometty, who succeeded Palmisano on Jan. 1. Rometty tapped former North American sales leader Bridget Van Kralingen to head Global Business Services. Claypool wrote to Van Kralingen earlier this month, complaining that little had changed since Kern proposed his remediation program: "The City of Austin continues to sustain substantial financial and operational losses resulting from two multi-million dollar procurements in which IBM has failed to provide the fully functional systems required out of three major system integration contracts we have with you." Claypool was referring to the billing system and also to an inventory management system IBM was supposed to build for the city. "More than a year later, the City is still suffering from serious, continuing issues in getting the system running and using it to conduct routine City of Austin business," Claypool wrote. As for the billing system, Claypool told Van Kralingen that Austin would hold back $3.8 million in payments to IBM until the situation is righted. "We are extremely disappointed and continue to have serious concerns about the quality of service we have received from IBM to date, which could impact any future contracting procurements," Claypool wrote on Feb. 10 of this year. In an interview this week with InformationWeek, Claypool said that, despite his harsh messages to IBM, he hasn't given up on resolving things amicably with Big Blue. "They have been working with us to provide a solution that meets the requirements," Claypool said. "They have been responsive. I would characterize it as, in their words, 'incremental response.' We would like to see a faster response." Claypool said there's no single root cause for all the problems with Austin Energy's billing system. "It's very complex. It's got 73 different interfaces," he said. "The teams have been working very hard through nights and weekends to figure out all of these complexities and integrations, to make them work with all the systems." Claypool was not Austin Energy's CIO at the time the utility drew up the contract with IBM. In the future, the utility's outsourcing contracts will include much more stringent penalties for non-performance, he said. "The penalties around specific areas like system availability need to be much stronger," he said. An IBM spokesman would say only that "IBM is working with Austin Energy to resolve the situation as quickly as possible." Question: This week we will discuss an article from Information Week with a rare look inside a failed (or failing) IT project at Austin Energy, the 8th largest public utility in the U.S. It's interesting that Austin Energy publicly aired its problems. Everybody wants to publish a success story, but in reality many projects have problems, only you rarely hear about them. I suspect that the current CIO is not the one who made the decision to go with IBM and then negotiated the contract. Read the article and answer the following questions: 1. What do you believe went wrong from an IT Governance / Project Management perspective? 2. What do you think are the underlying issues, and how could they have been prevented? 3. How can Austin Energy recover, short of walking away? Powered by WPS Office O