UPDATE: There was another outage on Friday, March 24 for a few hours. Another response from Intuit on 3/25/11 is posted below.
Intuit has issued the following in response to the recent service outage affecting QuickBooks Online, QuickBooks Merchant Services, QuickBooks Online Payroll and other Intuit services which were down on March 21 and 22, 2011.
A Letter from Kiran Patel- 3/24/11
To our customers,
I apologize for the frustrations caused by our service outages and for letting you down this week. We didn’t meet our own high standards for dependability and customer service. There is no excuse for this performance.
We want to help those affected by this outage. If you need further assistance, please contact us at the following numbers between 8:00 a.m. and 5:00 p.m. Pacific time, Monday through Friday:
- QuickBooks Online: 800-286-6800
- QuickBooks Online Payroll: 888-537-7794
- Intuit Payments Solutions: 800-558-9558
What Happened 
Over the course of two days we had service outages due to two separate errors made during scheduled maintenance.
In both cases we made network configuration changes that inadvertently blocked customer access to a portion of our servers. The initial disruptions lasted just a few minutes while we undid the change. But in both cases, restoring full service took longer than we would have liked. After fixing the problem, a surge in traffic overloaded the servers when we restored connectivity. We then decided to restore service at a purposefully measured pace to closely monitor the system and prevent another overload.
Be assured that there was no security breach or attack on our servers. At this time, we do not believe there is any damage or loss to your data.
Our commitment to you
We understand the important role our services play in your business. And we take that responsibility very seriously.
Again, we apologize for the disruption to our service and are working to improve the resiliency of our systems.
We appreciate your business and your patience. We will work hard to regain your trust.
Sincerely,
Kiran Patel
Executive Vice President and General Manager
Intuit Small Business Group
*********************************
UPDATE: Intuit Response on 03/25/11
This was a disappointing week, both for you and for Intuit. And yes, that may be an understatement.
You’re probably aware of how several of our online small business products weren’t available when you needed them. For many of you, it goes beyond disappointment and beyond frustration. You’re wondering how and why this happens, and what we’re doing to fix it – all legitimate questions.
Let’s start with an apology. Quite simply, this shouldn’t happen and there’s no excuse for this performance. We didn’t meet our own high standards for dependability and customer service. If your business was affected by these outages, we want to do right by you. If you were affected, call:
- QuickBooks Online: 800-286-6800
- QuickBooks Online Payroll: 888-537-7794
- Intuit Payments Solutions: 800-558-9558
What happened
On Monday and Tuesday, March 22-23, we had service outages triggered by separate errors made during scheduled maintenance. On Friday, March 25, we lost those services again. We’re still working to identify the cause of that last disruption.
In the first two instances, blame human error. We changed our network configuration and inadvertently blocked customer access to a portion of our servers. Those initial disruptions lasted just a few minutes while we undid the change. But in both cases, restoring full service took longer than we would have liked. After fixing the problem, a surge in traffic overloaded the servers when we restored connectivity. We then decided to restore service at a purposefully measured pace to closely monitor the system and prevent another overload.
On Friday, the same services went down. We restored them much more quickly, but are still trying to determine why that happened. In all cases, we know there was no security breach or attack on our servers. And we know that your data was not lost or damaged.
What we’re doing
Explanations and apologies don’t mean much if we don’t fix the underlying problems. With our services restored, we’re going to get to the bottom of this and solve it.
Here’s what we’re doing:
- We’re building resiliency and redundancy into our new products and data centers, so we can keep your services up and running in the event of a failure with minimal downtime.
- We’re improving the way we operate our data centers to reduce the risk of human error.
This is a work in progress. When it goes right, no one notices. When it goes wrong, it’s obvious. We’re committed to doing this as quickly as possible and, I hope, with no disruption to you and your business. If that happens, we’ll work to restore services quickly while staying in touch with you along the way.
As I said previously, we understand the important role our services play in your business. And we take that responsibility very seriously. While we do apologize, we want to do much more. We’d rather provide you the great service you expect. And we will. We appreciate your business and your patience.
Sincerely,
Kiran Patel
Executive Vice President and General Manager
Intuit Small Business Group