Open letter to Microsoft

Thanks for patching your operating system finally. I realize it must have taken some time to track down the hole and patch it properly and I thank you for spending the time to fix it. However, when we patched our webservers, your patch broke a good number of them. To make it worse, your "support staff" could not fix the problem because they didn't know what the problem was. "Perhaps it's this, or that, try this one." We had to have one of the more competent system administrators get on the webserver and figure out what your patch broke. The funny thing was that the fix we applied to fix one of the applications also fixed all the other webservers that the patch broke.

Did you really regression test the fix? I know that not all the web applications are running .NET - do you? The web applications that broke are the older .asp code that hasn't changed in a while, and frankly, some people cannot rewrite their website every 2.5 years to adhere to the latest guidelines coming out of Redmond.

The first rule of patches should be "do no harm". If the harm is part of the patching process, make sure your support team is aware of it or document it along with the patch. We don't like surprises at 1am. Patching dozens of servers and regression testing them overnight is a bitch. Please be aware that one of the reasons your ears ring in the early hours of a patch evening is because we have to roll back patches and leave our servers vulnerable for some period of time hoping that we (us and Microsoft) can figure out how to successfully apply the patch before the worm authors get their latest exploit out.